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abstract 


STAMM2R2  (System  for  Tactical  Assessment  of 
i^ultisource  Messages,  Sven  Radar)  is  an  experimental  prograr 
created  as  part  cf  an  investigation  intc  methods  cf 
correlating  information  in  the  naval  environment.  This 
thesis  is  an  exploration  intc  the  application  cf 
artificial  intelligence  to  the  tactical  situation 
assessment  problem  and  intc  various  evaluation 
methodologies  for  STAMMJR2.  Included  is  an  overview  of 
cr.e  of  these  experiments,  using  the  facilities  cf  the 
Naval  Postgraduate  School  Command,  Control  and 
Ccnmunicat lens  Laboratory  and  the  Naval  Ccean  Systems 
Center,  San  Diego,  California. 
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A.  EACH?.  CUM? 

STAh,ivIR2  (S/ste^  for  Tactical  Assessment  cf 

N'ultisource  Messa£es,  Iver.  Radar)  is  er.  experimental 
computer  ] rosrar  created  as  part  cf  an  investigation 
into  new  methods  of  c  or  re  la  t  in.-:  information  available  in 
a  naval  environment  in  response  to  tne  tactical  situation 
assessment  i?SA>  problem.  It  was  written  by  the  Tactical 
Comrana  and  Control  Division  of  the  Naval  Ocean  Systems 
Center,  San  Dlegc,  California.  STAKM2P.2  has  served  as  a 
testbed  for  explorations  of  applications  of  rule-based 
artificial  intelligence  inference  systems,  witn  a  general 
end  flexible  approach  to  the  ranee  of  acceptable  inputs 
tc  the  system. 

In  practical  application,  information  ir.  a  oasic 
forrat  is  received  from  a  variety  of  sensors.  This 
information  is  processed  within  c  network  of 
interconnected  rules  or  conditional  statements  whicn 
may  be  satisfied  by  this  information ,  even 
responding  to  information  in  cases  involving  varying 
degrees  cf  uncertainty.  STAKMR2  is  capable  cf 
performing  deductions  based  or.  this  received  information 
tc  speed  the  decision  process  in  a  rapidly  changing 
environment . 
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Before  we  discuss  the  specifics  of  STAKPERZ  ar.d  a 
method  of  testing  the  utility  of  this  decision  aid, 
tack? round  information  will  te  presented  to  set  the  stage 
for  the  why,  where,  end  how  such  a  system  was  developed. 


3.  OBJECTIVES 

There  are  three  principal  objectives  for  this  thesis: 

1.  Identify  the  charac teris t ics  of  an  artificial 

intelligence  prcgra-,  with  an  overview  cf  the 

applications  to  a  specific  military  situation. 

2.  Pronose  various  levels  of  evaluation  methodologies 
for*  STAKPER2 . 

3.  Present  the  results  of  an  experiment  conducted 

with  STAMPERS  using  one  of  the  methodologies 

presented . 


II  .  APT  I  r  I  C I  AI  I.MZIIIGZKCZ 

A.  LZHNITICN 

Artificial  in  tel  1  i  series  (  aI  ;  is  defined  as  "the  study 
cf  ideas  which,  enable  computers  tc  ic  the  things  tnat  make 
people  seer  intelligent"  rRef.  lj  end  "the  science  of  making 
rachines  do  things  that  would  require  intelligence  if  done 
cy  ren ’  [Ref .  2] . 

The  intricate  processes  from  which  these  ’things 
have  evolved,  such  as  learning,  growth,  or  ratu ration 
are  bypassed  in  the  artificial  intelligence  program  tc 
create  a  facsimile  of  human  thought  and,  subsequently, 
iecisicr.-mading.  But  the  bypassing  is,  in  reality,  a  very 
careful  compilation  of  tne  elements  which  are,  or  seem  tc 
be ,  parts  c  f  the  huma  n  thought  patterns  and  strategies 
leading  to  decisions.  That  is,  human  thought  is  analyzed 
and  dissected  into  discrete  elements  which  may  be 
recombined  artificially  to  mimic  human  intelligence. 

Al ter na t ively ,  differing  patterns  or  methods  are 

explored  to  uncover  new  routes  which  could,  in  fact,  lead  to 
similar  results. 

B.  DISCUSSION 

To  imagine  tne  thought  process  as  a  branched  tree, 
with  decisions  made  at  each  juncture,  or  node. 
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describing 

numan 
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actual 
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must 

consider 

many 

interconnected 

paths , 

with  multi  cl  re 

ctioaal 

flow 

through 

those  paths. 

A  c  c 

nt r ol  mechanism 

must 

also 

be  considered  which 

carries  a  thought  throu6h  to  conclusion  and  minimizes 
retraced  or  r.cn-prcductive  rcutes. 

a  goal  of  artificial  intelligence  inquiry  is  to 
duplicate  this  complex  structure  in  a  corputer  program.  A 
difficulty  lies  in  the  multi-dimensional  nature  cf  the 
thought  process  which  requires  equally  complex 
investigation.  Ey  examining  the  problem  from  the  viewpoint 
cf  discrete  elements,  it  ray  he  possible  to  find  the 
combination  which  accurately  reflects  a  pattern  which  net 
cnly  duplicates  what  happens  in  the  case  of  a  specific 
problem,  but  also  a  general  case  which  may  be  applicable  to 
ether  prcolems. 

If  STAfcl“!IR2  can  duplicate  the  reasoning  process  of  a 
decision  maker,  with  a  aata  base  and  memory  consistent  with 
the  situation  presented,  then  it  is  possible  to  use  this 
artificial  intelligence  program  as  an  "intelligent 
assistant"  in  the  tactical  situation  assessment  problem 
which  will  be  discussed  in  the  next  section. 

The  computer  is  the  "ideal  experimental  animal.' 
Infinitely  patient  and  easy  to  care  for,  the  computer 
requires  structured  instruction,  composed  of  discrete 
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elements,  which  it  combines  as  programmed.  The  combination 
and  recombination ,  done  at  speeds  sufficient  to  expose 
■unproductive  patns  rapidly,  can  provide  a  practical  test  of 
the  precision  with  vhicn  the  experimenter  defines  his 
thought  model  [Ref.  3].  Additions  or  deletions  which 
tne  experimenter  may  hypothesize  as  crucial  (cr  useless)  tc 
the  final  result  of  his  pattern  may  he  evaluated  with 
relative  ease.  Flaws  or  conceptual  mistakes  which  ray 
exist  are  likely  tc  be  detected  in  the  computer  because 
the  computer  generally  will  not  accept  ambieuity  cf 
instruction  and  may  simply  cease  execution  cf  the 
program.  However,  a  program  designed  tc  answer  a  certain 
sort  cf  question  does  not  guarantee  a  correct  answer 
[Ref .  4]  . 

The  applicability  of  computers  and  programming  is  not 
tne  emphasis  cf  artificial  intelligence.  However,  the 
unambiguous  nature  of  the  program  is  mentioned  to  reinforce 
tne  ”  transparency"  cf  the  actual  machine  and  focus  cn  the 
precision  of  understanding  human  thought. 

Using  artificial  intelligence  techniques,  research  has 
been  done  to  explore  vision,  problem  solving,  language,  and 
neurosis  [Ref.  5] .  Some  of  these  models  have  been 
successful  in  predicting  behavior  under  specified  conditions 
while  others  have  been  at  least  of  metaphorical  value  in 
helping  t c  understand  the  possible  mechanism  cf 
intelligence.  Ey  representing  thought  and  knowledge,  in 
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general ,  in  a  useful  and  flexible  manner  it  is  also  possible 
tc  arrange  an  interaction  between  a  human  and  tne  machine  in 
c  task  oriented  environment. 

inovledge  is  defined  as  a  collection  of  facts,  the 
state  cf  knowing,  or  all  that  has  been  perceived  by  the 
human  mind  [Ref.  6] .  Knowledge  ray  be  in  human  or  machine 
•’•emery  or  in  a  data  base.  It  is,  however,  also  in  the 
actual  procedures  that  operate  on  or  by  reference  to  the 
data.  Tc  store  disjointed  facts  in  computer  memory  or  in 
a  data  base  from  which  the  labelled  information  units  may 
be  retrieved  is  net  the  goal  of  artificial 
intelligence.  The  Identification  cf  the  desired  fact  icr  a 
location  code  by  which  that  fact  may  be  retrieved)  is  a 
problem  of  database  management.  Tc  discern  which  fact  is 
required  is,  however,  the  end  of  a  series  of  steps.  In 
the  problem  solving  case,  the  solution  process  is  the 
strategy  of  gaining  information  after  ”  a  forward  or 
backward  looking  reasoning  method'  [Ref  7]. 

Analogies  to  other  human  processes  exist.  For  example, 
radar  provides  target  information  in  a  manner  analogous 
tc  the  eye.  Data,  such  as  target  presence,  range,  and 
bearirg  may  be  useful  to  the  operator  as  well  as  in  a  form 
which  can  be  compared  tc  characteristics  stored  in  computer 
memory . 

The  specific  goal  of  STA(^^ER2  is  to  practically  combine 
this  sensory  data  with  the  problem  solving  process  of 
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data  receipt,  corparison  ana  retrieval  usin*r  the  knowledge 
stored  ir.  the  artificial  intellie.er.ce  prcsrraT. 


Ill . 


lACTICAL  SITUATION  ASS  TaSMv  T 


ITANN&RZ 


TACTICAL  S ITUATI O  ASSISSri’NT 


In  the  naval  context,  the  atility  cf  a  commander  tc 
accurately  assess  his  military  situation  rej.uirec  decisions 
about  his  environment.  While  all  decisions  need  not  be  rade 
cn  an  immediate  basis,  certain  cf  then,  called  tactical, 
determine  his  immediate  course  of  action.  In  ar  extreme 
example,  the  decision  tc  launch  a  defensive  wea per.  is  based 
on  e  judgment  that  such  an  action  is  required  based  on  the 
commander 's  evaluation.  A  staterent  of  the  tactical 
situation  assessment  ‘TSA)  prcbieT  includes  ccr.s  iderat  icr. 
cf  the  information  available  tc  the  decision  raker. 

Ir.  crier  tc  assess  a  military  problem,  the  evaluator 
draws  upon  experience  and  training,  after  having  considered 
the  situation  presented  tc  aim.  The  assess~ent  cf  the 
tactical  situation  is,  however,  highly  dependent  upon 
factors  associated  with  the  decision  maker  himself. 

The  experience  upon  which  a  military  decision  maker 
draws  is  uniaue  tc  him  alone,  although  there  is 
certain 1 y  a  similarity  of  career  patterns  and  situations 
common  to  many  individuals.  Unique  tc  that  individual  is 
the  ouallty  of  his  recollection,  his  emotional  cr  mental 
state,  the  perspective  of  the  individual,  and  the  length 


15 


cf  time  elapsed  since  t&cse  events  frcm  which  the 
experience  was  gained. 

The  actual  aecision  raker  in  the  operations  center 
the  Ccmtat  Information  Center  or  CIC)  cf  a  N'avy  ship  is 
the  Tactical  Action  Officer  (TAC).  During  the  period  cf 

the  Vietnam  conflict,  tne  rcle  cf  the  Captain  cf  a  ship  as 
final  fighting  authority  was  altered  somewhat  frcr  the 
traditional  rcle.  The  Tactical  Action  Officer  concept  was 
proposed,  implemented  in  Navy  Regulations,  and  exercised  in 
ccmtat  C?.ef.  8].  The  concept  tasicly  states  that  a  trusted 
subordinate,  trained  end  drilled  in  defensive  doctrine 
and  procedures,  acts  as  weapons  release  authority  in  the 
temporary  aosence  cf  the  Captain. 

The  specialized  formal  training  for  TAOs,  having  as 


prerequisites  proven 


maturity  and 


cnerat  icnal 


experience,  lasts  approximately  six  weeks.  The  student  is 
exposed  to  infornation  concerning  U.S.  weapons  systems 
end  those  of  jotentiel  enemies.  After  this  is 
committed  to  memory,  ’  standard"  tactical  doctrine  is 
exercised  in  a  series  cf  increasingly  complex  scenarios. 
The  scenarios  are  r re  sen ted  in  sequences  allowing 
questions  tc  te  answered  based  cn  in  format  i  cr.  normally 
available  to  the  TAC  (from  the  training,  experience, 
tactical  putl  icat ions  .sensors  ,  and  Intelligence  sources). 

The  TAG  becomes  the  focus  of  the  Combat  Information 
Center.  All  serscr  reports  are  displayed  tc  him,  all 
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r esseges  from  external  sources  are  r?i  ver.  tc  him  directly  or 
ir.  summarized  form.  He  can  stand  alone,  cn  r.is  cwr. 
ship,  or  be  electronically  linked  by  radio  tc  TAC-s  on 
other  snips,  each  =  eir.<?  three yn  similar  procedures  ar.i 
actions.  This  defines  a  sphere  of  knowledge  in  which  the 
TAG  operates,  ana  can  generally  be  delineated  by  the  ran^e 
cf  his  sersers.  A  ccrreleticn  function  is  required  when 
the  TAG  receives  'external'  reports  from  sensors  or 
platforms  ether  than  his  own — in  effect  from  a  sphere  cf 
larger  'radius. 

Assumptions  invclvea  in  reducing  many  TAG  activities 
to  computer  form  include:  a)  the  basic,  repetitive  mass 
of  information  can  be  reduced  to  a  computerized  database; 
b )  standardized  procedures  for  data  retrieval  ar.i 
correlation  can  be  similarly  reduced ;  c ) the 

sensor/ intelligence  information  can  be  presented  in  a 
standard  format;  and  d)  the  tncu^ht  process  by  which  the 
TAG  functions  ray  be  artificially  reproduced. 

r.  SIAMIES 

STAt'M°.2  is  a  revised  version  cf  3TAVVZ?.,  a  ?ystem  fer 
Tactical  Assessment  of  f-ulti  source  (■  e  ssa^'es ,  Iver.  Hadar.  It 
was  created  as  part  cf  an  investigation  cf  correlation 
methodologies ,  and  served  as  a  testbed  for  explorations  of 
applications  of  rule-based  inference  systems  to  tne  tactical 
situation  assessment  !'TSA)  problem.  STAy^IP.  concentrated  cr. 
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the  specific  task  of  merchant  shir  discrimination  i'rcr  all 
ccntarts  reported  by  radar  and  external  messages  _?.e? .  j]  . 

STi-WIHZ  is  sl  organizer  of  information.  It 
collects  information  cy  receiving  •r-essas~es  ar.  i  ser.scr 
reports  (radar,  electronic  support  neasrres,  ar.d  scr.ar', 
and.  organizes  this  raw  data  i  ate  graphic  displays  and 
textual  commentary  to  aid  ir.  tactical  situation 
a ssessreat.  Through  the  use  of  specified  rules,  the 
system  combines  tais  information  tc  draw  ccnclvslcr.s 
abort  the  situation  in  the  vicinity  of  the  home  ship.  Tncse 
combinations  are  reflected  in  both  the  STAH*‘~?.2  display  a-d 
correct ary.  The  system  is  available  for  tne  examination  of 
raw  data  as  well  as  information  about  w.ny  and  hew  the 
conclusions  were  reached. 

STAI^MP.2  deals  with  information  cn  a  real-tire  basis. 
As  a  message  is  received  from  a  sensor  or  information 
source  in  3TA^N;~H2-reada'ole  format,  it  becomes  part  of 
the  data  base  cn  which  the  rules  operate.  This  process  is 
complicated  by  the  fact  that  the  information  is  suspect,  and 
its  arrival  ray  net  be  ir.  chronological  order. 

Althou^n  ST.1KKIB2  was  developed  in  response  tc 
I.avy-  sponsored  investigation  of  the  TSA  problem,  the  U.S. 
a  cry  .nas  conducted  similar  investigations  in  the 
field  cf  military  intelligence  analysis  ]?.ef  .  1C], 
rasea  upon  a  detailed  stuuy  cf  the  analysts'  role, 
mettcls,  and  thought  processes  in  intelligence  production, 
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it  may  be  pcs. 

:  V  ^  Q 

tc  St 

rear-iine 

intelligence  analys 

is  . 

in  essence, 

tni  $ 

p  r  r  c  e  s 

s  used 

by  the  analyst  is 

the 

r ethodolo ry 

f  tot 

wi.  i  ch 

the 

AI  proiran  right 

he 

developed  af 

ter 

a  des 

criptive 

red  el  is  cr- a  r.i  zed 

a  r.i 

exe rc i sad  . 

The  manner 

i  a 

which 

STAb  MSH2 

functions  may  best 

he 

described  by  analogy  to  the  medical  diagnosis  systen  called 
f-'YCIN  [Eef.  IX].  f-YClN  is  a  problem  solving  systerr  by 
which  a  r*'.ysician,  responding  t  c  «  v.e  r  ies  f  r  c"  the  pregram , 
receives  assistance  in  identifying  and  treating  bleed 
bacterial  infections.  The  basis  fer  the  expertise 
resident  in  '-‘TCIi.  is  the  knowledge  obtained  frer 
interviewing  physicians  ar.d  experts  and  stored  in  ever  3?? 
"productions".  Productions  are  memor./  structures  or.  which 
the  pro^rcr  operates  and  are  in  the  following  format. 


If  the  infection  type  is  primary-bacteremia , 

the  suspected  entry  point  is  the  gastrointestinal  tract, 
and  the  site  of  the  culture  is  cne  cf  the  sterile  sites. 


then  tnere  is  evidence  that  the  organism  is  bacteroides. 

Conceptually,  inference  rules  are  very  simple.  Every 
rule  attempts  to  retrieve  information  from  memory  and,  if  it 
succeeds,  constructs  a  new  assertion  lr.  re-cry.  If  all  the 
conditions  are  matched,  then  the  rules  are  said  to  ’fire". 
That  is,  all  the  acticr.s  which  are  defined  in  the  rule  are 
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men  carried  out.  These  actions  may  ce  modifications  tc  toe 
data  base,  ass  i  .-'nir.3  cf  confidences,  and  print  ins  cf 
formatted  descriptions. 

In  a  sinple  rule  interpreter,  the  system  maintains  a 
list  cf  rules,  each  with  conditions  vnicr.  pertain  to  a 
central  data  base,  whicn  is  constantly  feeing  upaated.  At 
renlar  intervals  the  system  attempts  tc  fire  each  rule; 
that  is,  it  checks  to  see  if  the  rule  conditions  are 
currently  satisfied  in  the  data  base.  If  ere  cf  many 
•conditions  is  not  satisfied,  a  rule  ray  fail  to  fire  and,  in 
tnis  case,  any  partial  work  dene  in  satisfying  the  rule  is 
lost.  In  addition,  each  rule  is  retried  at  the  tme 
interval  witnout  regard  tc  the  nature  cf  me  intervening 
charges  to  the  data  base,  duplica  ir.5  results  where  no 
changes  to  the  data  base  nave  occurred. 

Because  medical  diagnosis,  even  by  experts,  involves 
varyirj  decrees  of  uncertainty,  MYCIK  is  written  with  a 
so-called  certainty  factor  for  each  ccnclusicn.  The 
physicians  upon  whose  information  the  productions  we re 
established  were  also  asked  to  provide  "strength 
values",  or  probabilities  cf  accuracy  and  confidence, 
for  their  assertions.  This  introduces  the  ability  to 
deal  with  uncertain  evidence  into  the  artificial 
intelligence  system. 

The  productions  used  to  reach  a  conclusion  are  part  cf 
the  conclusion  set.  *'YCIN  answers  questions  about  how  and 
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v. ny  a  fact  was  established  cr  usee.  The  production  is 
recalled  i  by  nare  cr  nu"ter,  tc  shew  an  enu^era ti  cr.  cf  the 
facts  presented  in  the  premise  set.  Ser.eral  facts  can  be 
questioned  without  resorting  tc  the  entire  diagnosis  because 
each  production  “staris  clone”,  to  be  actively  examined  or 
quest ioaed .  In  this  case  tne  steps  tapen  to  arrive  at  that 
particular  fact  can  also  be  traced.  This  tracing  may  act 
as  a  training  aid  fer  the  user  in  addition  to  presenting 
alternatives  wnich  can  be  explored. 

Tie  critical  aspects  of  STAb'MRZ 's  design  ray  be  divided 
intc  four  parts.  These  are  -^e^cry  (tne  data  base),  rule 
interpretation,  explanation,  end  graphics. 

The  rules  cf  STAMPERS  consist  cf  conditions,  actions  and 
confidences,  with  conditions  and  actions  in  the  sere 
structure  as  assertions  with  allowances  race  for  variables 
which  are  "bound”  tc  a  rule  by  a  bind  in*  function  ir.  LISF, 
the  language  in  which  STAMMZF.2  is  written  [Ref.  12].  These 
variables  which  are  applied  tc  the  rule  conditions  will  be 
evaluated  to  see  whether  tne  condition  succeeds  or  fails. 
If  it  fails,  a  ’’suspension”  is  created  which  corresponds  tc 
the  remainder  of  the  rule.  As  rore  inforration  is  added  to 
the  data  base,  these  suspensions  which  can  use  the  new  data 
are  revived  and  continue  as  before  (either  corpleting  or 
suspending  again;.  A  suspension  contains  r.ct  only  tne 
remaining  part  of  the  rule,  but  also  the  bindings 
establis.ned  by  already  satisfied  ccnditirns.  If  any .  Ever. 


22 


wrier  a  condition  succeeds,  there  may  be  other  ways  for  it  tc 
be  satisfied,  so  a  suspension  is  left  behind. 

The  individual  rules  are  expressed  in  the  following 
f  orr . 


RL'IZ-NaM 
(CONDITIONS 
(l  Rendition  11  ) 
1  <ccnditicn  2  w  ; 


v  <ccr.diticn  ni-  ; ) 
ACTIONS 

(I  <actior.  1  ✓  ) 

(  <action  2>  ) 


(  <aotior.  m  >  } 

CON?  <a  number  between  -1  ar.d  +1' 

PaiNFCRP 

"This  is  a  sentence  describing  the  rule.") 

STAht‘SP.2  deals  wit;.  information  or.  a  real-t  i~e 
basis.  However,  this  process  is  complicated  by  tne  fact 
that  the  arrival  of  reports  ray  net  be  in  chronological 
order,  with  later  information  superseding  or  negating 
earlier  reports.  A  data  stream"  is  used  to  bind 
information  as  it  is  received  to  tne  rule  condition  which 
i t  satisfies . 

A  data  stream  ray  te  defined  as  a  secuer.ce  rf 
values,  existing  over  time  in  a  computation,  if  a  program  is 
executed  in  a  conventional  language,  then  the  history  of 
successive  values  of  a  variable  forms  a  stream.  Thus,  ir. 


contrast  to  static  data  structures  such  as  lists  ar.d  arrays 


where  all  the  elements  exist  at  cr.e  tire.  streams  -re 
cyr.arir  data  structures,  ana  are  addressable  objects  i 
LISF.  Cnly  the  rev.  aata  received  need  he  compared  tc  the 

rules.  New  matches  are  added  to  the  end  of  the  stream.  r. 

temporary  "freezing"  cf  the  action  specified  by  an  assertion 
occurs  until  after  the  new  lata  is  read  onto  the  appropriate 
'strears.  The  strear,  then,  allows  the  rapid  review  cf  stored 
infer ration  without  the  penalty  cf  complete  review  cf 

potentially  irrelevant  data. 

I  r.  S  T  AVMH2 ,  confidence  is  creviced  in  the  rules  by  the 
creator  of  the  rule.  It  is  calculated  dynamically  or 
request  at  the  tire  when  it  is  displayed  to  the  operator 
fcllcwir?  a  rule  rate:;  and  is  net  stored.  Each  assertion 
will  have  as  its  confidence  the  con  bination  of  the 

confidences  cf  the  rules  and  assertions  which  offer  evidence 
for  it.  k  confidence  calculation  is  highly  dependent  upon 
t.ne  connective  t  n rough  whica  the  rules  and  assertions  are 
combined.  An.  ANI  connective  will,  fer  example,  display  the 
confidence  of  the  snsllest  value  of  all  the  rules  ccrbined. 
That  is,  the  conclusion  nay  never  be  stronger  tnan  the 
weakest  piece  of  supporting  evidence. 

Turing  execution  of  S?AiyMS32,  the  user  will  see 
the  following  cycle  repeated  as  long  as  messages  end  reports 
are  received  into  the  ressage  input  file: 

1.  A  message  revert  is  received,  the  user  is  inferred, 
ana  the  critical  inferration  in  the  message  is  printed 
fer  reference. 
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c.  . 


\ 


A  display  ,  shcwin.-  the  area  situation  with  the  new 
information,  is  uravn.  The  user  ray  manipulate  trie 
i~a..e . 

The  system  makes  sore  commentary  or.  the  conclusions  it 
oar  ream,  or  the  oasis  of  the  rev  information. 

4.  If  a ~j  conclusions  were  reacned,  tr.e  user  is  river  the 
opportunity  tc  juery  the  system  about  the  contents  of 
its  hata  base. 

The  message  text  is  in  the  basic  format  of  the  received 
ressa^e.  A  contact  nare  (if  available),  is  displayed, 
followed  by  location  and  time  cf  message.  The  user  m&y  pause 
tc  examine  this  message  or  select  the  graphic  display  rode. 

Graphical  support  for  STAMPi’HZ  is  provided  by 
TS?TA,  a  software  package  developed  at  the  '.aval  Ocean 
Syste-1'  Center  Specifically  for  tactical  situation 

assessment  [Ref.  13].  The  TSPLA  system  is  a  collection  of 
iCRTR  Af.  subroutines  that  allow  storage,  retrieval,  and 
cisplay  of  ship  ana  aircraft  tracks.  This  display,  with  the 
capability  tc  shew  -acs  with  latitude  and  longitude,  ~a y 
ce  r  - !  pi:  la  ted  by  the  user  tc  vary  the  scale  as  desired.  As 
sccr.  as  a  ^essa^e  is  received  which  contains  a 
demonstrable  assertion  t  such  as  a  contact  at  a  specific 
location)  t r.e  display  will  appear  on  the  display  screen 
■if  a  display  is  available  and  selected  in  the 
ini t i a li za t ion  process).  Zxamples  cf  basic  function  keys  are 
are  listed  below. 

V  ramify  about  the  cursor  oy  a  factor  of  2. 

?.  reauce  about  the  cursor  by  a  factor  cf  2 


r  e  r.  t  c  r 


1 


V  cer.tcr 

t;iS  view  at  cut  tne  cv. 

rc  c  r 

p  c  s  i  t  i  c  ~ 

;  set  typ 

e  sice  to  smallest  or. 

The 

T*- ?: t ?. c *•; i :c  -yi4 

C  reivrs 

tc  command  mode 

ore  ietailei 

functions  exist,  with 

the 

fvll  listir.r  in  ••CSC 

1  e  coni  cal  recurrent  252.  T ns  user  ray  return  to  the 
cc-rend  level  to  begin  tte  explanation  and  i.uery  procedure. 

The  explanation  s/ster  ..  rovides  tve  prirar/ 
capabilities:  retrieval  cf  me-'cr.y  and  inference  tracing. 

£.lth,“u^h  there  would  appear  to  be  little  difficulty  in 
retrieving  memory  ccr.tents  and  tracing  derivations,  the 
display  of  this  information  in  a  human  reaaable  format  is  a 
major  ccns  iaera ti  cn  .  Tc  make  the  user  interface  as  natural 
as  possible,  the  explanation  system  provides  a  query 
language  that  is  'in^Hsh-like".  This  language  is  an 
ext  rerely  limited  version  of  English ,  which  includes  only 
certain  types  of  questions  and  rethods  cf  phrasing  those 
-uestioos.  However,  the  language  was  designed  sc  that, 
while  limited,  it  is  suificient  to  cover  the  user's  needs 
without  making  its  shortcomings  apparent.  The  LIS?  fvnetier 
.-.SEUSE?,  which  features  recognition  ar.d  prompting,  is 
coTbined  with  a  "p  rettyrri  nter"  (to  add  words  like  is  ar.d 
"of"  to  memory  ccr.tents)  to  simplify  the  user  interaction. 

A  summary  cf  the  queries  which  the  STAivMR2  user  can 
ask,  with  .general  examples,  follows.  following  the 
initial  query  word,  only  these  inputs  shown  will  be  allowed 
by  the  ASKUSIR  function.  The  connective  words  (in  lower 


case)  will  lie  provided  by  the  "prettyprinter"  fRef.  14]. 
Che  words  ir.  parentheses  may  t e  inserted  by  the  user  as 
desired . 

«fSAT  Forrat:  i K AT  is  (THE  AN  A)  CHELATION'  (CF) 

< I TZf.> 

Ixarple:'*HjvT  is  the  COURSE  of  SI5HTING2 
This  every  provides  cr.e  c?  several  ouesticn 
formats  for  a  Siting  about  entries  in  the  data 
base. 

13  Fermat:  IS  'TEE)  <ITEP>  .A  AN  THE ) <HSLATION> 

(OF)  <ITZI*> 

Example  :  I S  RAIAR  the  SOURCE  cf  S IGFTING2 
This  question  ferr  allows  the  user  to  ask 
about  specific  entries  in  the  data  base. 

TILL  re  about  lorrat:  TILL  me  about  <ANYTHINS> 

Example :  TILL  me  about  SIGHTING* 

This  is  the  most  flexible  nuery,  and  allows 
the  user  to  ask  rules,  categories, 
relations,  or  specific  facts. 


V  HIRE 


Format:  WHERE  is  <.C3JECT>  cr  WHERE  was 
<'CEJICT>  at  <TIKE> 

Example:  WHERE  is  CONTACT? 

With  this  ccrrand  the  user  can  as>  about  the 
position  cf  a  platform,  merchant  lane,  cr 
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st on*  at  the  present  tire,  or  in  the  future 
or  past. 


WET  Fermat:  WHY  is  <ASSERTICN> 

Exerple:  WET  is  AZ034E 

With  this  command  the  user  car.  find  cut  the 
primary  or  immediate  reasons  that  STACKERS 
used  tc  conclude  any  assertion.  All  the 
rules  involved  in  the  decision  will  be 
displayed  . 

EC'*  Format:  HCW  does  rule  <RUL2>  apply  to 

< ASSERT  I CN> 

Example:  EOV  does  rule  ID-LAN5  apply  to  A334 
This  query  allows  the  user  to  find  out  what 
assertions  or  facts  were  involved  in 
permitting  the  rule  to  help  conclude  the 
given  assertion. 

WHOSE  Fermat:  WHOSE  <RELATION>  is  <ITEM> 

Example:  WHOSE  TYPE  is  MERCHANT 

This  query  acts  as  a  partial  inverse  tc  the 

WHAT  «uery. 

WHO  Fermat:  WHO  is  \THZ  AN  A '  <RILATICN>  (OF} 

<ITIK> 

Example:  WHC  is  INS  ICE  IAN  El 
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Ili .  IXFir.IMNTAL 

This  section  presents  3  discission  c  f  exper i~er ta i 
methodology  which  may  be  applied  in  the  evaluation  cf 
STANMP.2.  Following  an  overview  of  general  issues,  and  a 
discussion  cf  general  levels  c  f  experimentation,  three 
specific  stages  of  experiment  are  presentee.  From  these,  one 
has  been  chcsen  tc  oe  carried  cut  as  a  sample  experiment. 

The  design  of  an  experiment  is  based  on  the  question  tc 
be  addressed.  An  experiment  tc  compare  systems  with 
different  speeds  of  message  receipt,  for  example,  will 
include  a  message  generator,  an  interval  for  testing,  and 
the  means  to  measure  differences  in  total  messages 
received.  The  criteria  for  "better"  m ay  be  the  higher  number 
cf  ressages  received  ir.  a  certain  time  or  the  shorter  time 
needed  tc  receive  a  certain  number  of  messages.  These 
objective  Teasures  cf  performance  are  established  prior 
to  the  experiment,  with  sufficient  flexibility  to  present 
the  data  tc  satisfy  the  question. 

Ir.  more  corplex  evaluation,  the  questions  tc  be 
answered  may  be  difficult  to  define.  If  the  goal  of  the 
experiment  is  to  find  the  number  of  messages  received  and 
understood  during  a  definite  time  interval,  t.ne  criteria  for 
understanding  must  also  become  a  part  of  the  experimental 
procedure.  Interpretation  of  the  results  cf  such  an 


experiment  t r.er-  becomes  dependent  cr.  an  "acceu ta tie ’  level 
r f  i Me rs tc nd in.*,  ?r.  acceptable  number  of  messe--es,  or  a 
combination  of  the  two.  Tais  ray  then  be  ar.  area  where  the 
objective  measures  wive  way  to  less  precise  but  equally 
useful  subjective  evaluation. 

In  order  tc  test  the  suitability  of  an 
artificial  intelligence  system,  measures  of  performance 
-ray  be  difficult  tc  establish.  Tne  components  of  the 
program  may  be  individually  examined  (the  structure, 
execution,  results),  or  the  overall  ’'improvement"  in  a 
parameter  of  the  function  which  the  program  is  designed  to 
assist  may  be  tested.  However,  tc  show  that  tne  system 
world  respond  tc  a  euest i cr.  cr  situation  as  uuickly  as 


a  human  decision  maker,  given  that  the  situatio: 


were 


constrained  tc  wnat  the  program  and  thp  human  knew  and  tne 
information  flow  was  at  "normal"  speed,  may  prove 
nothing  mere  than  that  ccrtuters  are  faster  than  humans. 
Considering  the  difficulty  in  establishing 
meaningful  measures  of  effectiveness,  subjective  evaluation 
of  the  capability  and  the  utility  of  STACIES:,  by  tactically 
orientea  Individuals,  ray  show  the  greatest  premise  as  an 
evaluation  technique. 

Experimentation  ray  be  performed  ir.  stapes  r, arming  from 
simple  static  debug /valiaa  tioc  experiments,  whirr, 
concentrate  on  tne  capabilities  of  the  tecr.nrlcmy  a^d 
require  few  resources,  tc  more  extensive  technical 


evaluations  c  f  STAt-.v,?-52  with  a  dynamically  ch?  ";!w  data 
base  in  a  controlled  e  r.vi  rcnrent  .  This  could  include 
a/uamic  operational  evaluations  in  simulated  ccmmana  and 
control  environments  pitlir.*;  cones  ir.r:  forces  against  eacr. 
other  under  various  scenarios  us  ir.«-  one  of  the  warfare 
environment  sirulator  (tZS)  war  *ame  programs  [?.ef.  15]. 

t  o  1 1  o  w  i  r.  £  an  overview  of  three  possible  levels  of 
testing#  more  detailed  prccedures  for  these  levels  will  be 
oreser.ted  . 


The  basic  level  of  experiment  consists  of  ir.putir.r  into 
a  data  base  various  static  threat  scenarios  (a  snapshot 
view  of  tne  situation}  concerning  the  status  of 
'  riue’  ana  ’ Cran£e"  taslc  forces.  The  test  objective 
is  to  evaluate  the  capabilities  cf  STAMPS?.*  tc  read  the  data 
base,  signal  the  existence  of  the  threats,  and  form 
opinions  based  cn  specific  rules  selected  cy  the  test 
conductor)  concerning  other  information  ir.  the  form  of 
sensor  and  i r» * °1 1  ic  ence  reports. 


The  next  level 


experiment  is  conducted 


rore  realistic  command  and  control  environment  with  a 
dynamically  cnan.ei  r.^  data  tase.  A  detailed  scenario  is 
presented  via  tZZ  (or  its  variants),  p resenting  a  wide 


rar.re 


inputs  tc  S TAivf'.I?.2  under  char.*?in=:  circumstances 


to  probe  the  Limits  of  its  capabilities ,  e.r;.,  w.oat  threat 
intensity  is  ^euuired  tc  "overload"  STAVM?.2  , 


or  wr.at  is 


I 

tie  length  of  time  i'ror  tie  instant  trie  report  is  received 
until  STAMPERS  signals  a  conclusion. 

In  the  rost  complex  level  of  testing,  V.TS  is  again  used 
witn  Grange  and  Blue  task  forces.  These  forces  are 
roTpcsea  as  closely  as  possible,  witnir.  the  capabilities 
of  available  test  resources,  « i th  equipment  iser.scrs, 
weapons,  etc)  and  platforms  that  represent  current  military 
inventories .  Experimental  emphasis  is  shifted  from  a 
technical  evaluation  to  an  operational  evaluation  of  the 
value  cf  STAtv^Z?.2  to  tne  decision  maker.  The  objective 
is  tc  determine  if  STAKMr.c  improves  the  commander's  ability 
to  m.aae  rapid.  and  accurate  decisions.  Operationally 
realistic  scenarios  are  the  desired  norm.  Each  scenario 
sucvld  be  replicated  with  different  players  with  sc~e 
trials  having  the  Blue  forces  operating  with  S TANMP.2  and 
sene  without.  Operational  measures  cf  effectiveness, 
which  depend  on  the  scenarios  under  play,  are  selected  tc 
assess  the  operational  utility  of  ST-idlE?.  In  addition,  the 
players  and  the  umpire  team  are  subjected  to  pc s t-exe rc 1 s e 
interviews  and  questionnaires  tc  «et  subjective  evaluations 
cf  the  worth  of  STAMKIP2 . 


While  tne re 

are  no 

definitive 

bcur.da  ries 

between  tne 

levels  cf  the 

ferial 

experiments 

,  the  rar.me 

cf  renditions 

.•oust  be  designed 

for  scenarios  that 

are  Doth 

realistic  and 

informative.  The 

a  dual 

analysis  at 

each  level 

of  experiment 

•rust  take  into 

a cc  ciint 

the  limitations  of 

the  scenario 
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. enerator  r?.ef.  16].  Che  three  proposed  levels  of  experiment 
’*  i  1 1  new  oe  presented  in  detail. 

A.  STAGE-CNI  EXPERIMENTS 

1 .  C odec  tive 

Sta^e-cne  experiments  evaluate  the  capability  cf 
STAMMEE2  to  identify,  in  a  static  environment,  the  tactical 
situation  that  confronts  an  afloat  task  force  and  tc 
describe  the  situation  semantical iy  and  graphically.  This 
tyre  cf  experirent  establishes  the  type  cf  threats  that 
STAMMERS  car.  evaluate  and  assesses  the  timeliness  of  the 
warnings.  The  experiments  may  also  serve  to  provide  r  sdback 
through  which  STAMM ERZ  may  be  refined. 

2 .  Resources  Required 

a.  Technical 

The  STAMMER2  pro&ran  and  associated  computer 
uardware  are  required  for  this  experiment .  At  present, 
STAMMZR2  is  resident  or.  the  T  CPS  22  system  at  :.CSC,  Sen 
Tie?c,  California. 

b .  display 

An  alphanumeric  display  system  will  be  required 
for  players  and  the  umpire  team.  Ir.  addition,  a  graphical 
display  of  the  situation,  as  is  normally  eeneraten  ty 
V.-ES  rust  be  available  tc  compare  it  with  that  presented  ty 
STAMMR2. 
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c.  Scenario 

The  tactical  scenario  is  presented  as  a  series 

cf  operationally  realistic  aata  plates'  constructed  in 

advance  of  the  experi~er.t.  Each  data  plate  is  a  data 

snapshot  describing  a  tactical  situation  at  a  given  tire. 

«* 

The  plate  snculd  include  various  information  * course,  speed, 
sensor  reports,  intelligence)  available  tc  a  Tactical  Action 
Officer  tc  describe  tne  situation  fully. 

d.  Personnel 

The  personnel  required  for  the  experiment 
include  observers  of  the  output  (threat  assessors),  a 
scenario  controller,  and  a  test  director. 

e.  Support  Program 

In  addition  tc  scenario  generating  requirements, 
software  tc  measure  end  record  the  ti~es  at  which  the 
messages  are  output  and  the  tires  designated  by  the 
observers  is  required.  This  prcsraT,  based  on  tne 
nr.FaMT  TYPESCRIPT  feature,  records  ail  inputs  and 
outputs  tc  selected  terminals. 

2 .  Per  era  1  Context 

a  .  Coacep  t  and.  heed 

STAPIdLF.ic  is  being  developed  to  interact  with  a 
=.rapnics  display  system  to  support  the  commander's  decision 
processes.  In  order  tc  assess  the  value  cf  the  technology 
Me  rust  determine  its  capabilities  and  shortcomings.  Tor 
STW'vi?.£  tc  be  useful  tc  a  decision  maker  it  -ust  te  able  tc 
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evaluate  threats  'ey  a  comparison.  a r.d  collation  cf  rll-sm  rce 
reports  a r.i  display  the  situations  in  a  timely  manner. 


b .  General  Situation  and  Scenarios 

A  suitable  number  cf  data  plates  snould  be 
•-‘evelcred  tc  vary  the  tactical  sitraticr  significantly  ir 
both  the  complexity  of  the  situations  and  the  nature  and 
number  of  the  platforms  involved.  A  data  plate  should  be 
selected  at  random,  its  number  recorded  ar.d  the 
appropriately  stored  data  input  to  tne  file  which  STAf-ri?.2 
will  reed.  An  observer,  unaware  cf  the  contents  of  the  data 
plate,  ncnitors  the  contents  of  the  display  terminal, 
evaluates  the  information  received,  and  describes  the 


situations  as  he  sees  then,. 

4 .  Evaluation 

a .  Da  ta  ccl lect i on 

The  following  data  should  be 
each  trial:  the  data  plate  Her.  tificatio 

situation  assessment  cf  tne  evaiuatcr, 
situation  assessment  iiver.  by  3TA!“T1L?.2 . 

b.  Analysis 

The  primary  areas  of  concern  for 
experiments  are:  a  ccrparisot;  cf  the  STA.V!*"?.2 
tne  human  assessments  of  the  threats,  and  a 
cf  the  tires  required  for  S7AMH?2 
different  types  of  situations. 
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number,  the 
and  the 


the  ste^e-cne 
output  with 
de  termi  r.at  ion 
tc  process 
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Post-test  analysis  ull  exarine  t:.e 


sr.ee  t 

tc  ascertain  if  ST.i<vbZ?.2  accurately  evaluated  or  described 
and  displayed  all  phase-one  situations.  This  is  ccrpared 
with  the  evaluators'  ability  tc  do  the  seme  cased  or-  the 
same  date. 

c.  Anticipated  Results 

It  is  anticipated  that  3iAvb'iR2  will  si-’r.cl  the 
existence  of  threats  with  very  little  tine  delay  and  that 
the  situation  and  display  will  be  presented  accurately,  'or 
tr.cse  cases  where  complex  multiple  situations  are  present 
siru 1 te necr s ly  ,  the  tests  nay  indicate  an  overcrowding  of 
tne  display  and  a  lack  cf  tire  tc  exercise  the  logic  trace 
capatility  cf  STAMMR2 . 

c .  Comments  and  Srecial  Instructions 

Each  experiTental  subject  will  =?c  tnrrvgn  a  short 
STAhh'lRS  1  ndo  ct  ri  r.a  t  i  on  session  during  which  ne  is  .riv^n  a 
surrary  cf  tne  theory  and  implementation  cf  SIAM'1"*  R2  ,  a 
’threat"  briefing  tc  explain  the  scenario,  end  ar.  overview 
cf  what  is  expected  of  him .  Care  should  be  taken  tc  include 
in  the  data  plates  as  many  situations  as  possible  that 
STACMA2  has  been  designed  to  handle  in  realistic 
situations.  It  is  recommended  that  several  different 
operators  be  subjected  to  tne  complete  set  of  plates  and  te 
asked  tc  subjectively  evaluate  the  displayed  information  tc 
provide  information  whicr.  may  be  useful  ir.  the  desi*-r.  of 
follow-on  display  experiments. 
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The  stage-ore  experi  entaticr  diecussei  atc/e  will 
r.eed  to  be  carried  out  in  the  course  of  a  full  and  careful 
evaluation  of  STAiNMR2  as  a  tool  in  tactical  assessment.  Vor 
the  purpose  of  this  thesis,  however,  only  the  stage-two 
experiment  (as  discussed  below;  was  actually  performed. 

r .  STAG 2 -TWO  EiPFRI^ENTS 
1 •  Objective 

The  objectives  cf  sta^e-two  experiments  are:  tc 
assess  the  technical  capability  of  STa^IRZ  actions  on  a 
dynamically  changir.?  data  base  generated  by  creating 
threat  situations  through  the  exercise  of  VIS  (or  its 
variants'?  tc  determine  tne  limits  of  the  capability 
cf  STANMR2?  and  to  obtain  subjective  opinions  about 
STAi-PIRS. 

2 .  Resources  Resulted 
a  .  Techai cal 

The  S TAi".r*' IR2  program  and  associated  computer 
hardware  are  required  for  this  experi  ff  r.t .  ""he  Warfare 
Environment  Simulator  (WIS)  pro6rar  is  available  on  the 
TINEX  system  at  NOSC. 
b.  Display 

An  alpnenurerir  display  system  i«  reuuired  *',cr 
players  and  the  umpire  team.  The  GIMSCC  system,  a  color 
graphic  display,  may  be  used  tc  present  the  situation 
generate^  by  WIS  . 


which  is 


c.  scenario 

An  operationally  realistic  s  ce  na  ri  o  is 
generated  tc  test  STAdM?.2  in  a  tactical  situation  at  a 
liver  tire.  The  scenario  should  include  various 
inforration  (course,  steed,  sense r  reports,  intelligence) 
available  tc  a  Tactical  Action  Officer  tc  describe  the 
situation.  as  it  is  corrally  presented  by  *ZS  ,  via 

visual  display  and  status  boards.  The  evaluator  will  be 
allowed  to  interact  with  units  under  his  control”  by 
crderir.2  course  and  speed  changes,  and  enabling  various 
sensors  available  to  hi  it.  .  The  test  director  ray  insert 
various  elements  cf  intelligence  as  desired. 

4 .  Personnel 


The  personnel  required  for  the  ezperirent 
include  observers  cf  tne  output  (threat  assessors),  a 
scerario  controller,  and  a  test  director, 
e.  Support  Program 

In  addition  to  scenario  genera  tiny  re uui rerpr ts  , 
software  to  measure  and  recoru  tne  tires  at  which  tne 
ressages  are  output  and  the  tires  designated  by  the 
observers  is  desirable.  This  preyrar,  based  on  the 
AHPAFZT  TYPESCRIPT  feature,  records  all  ir.nv.ts  and 
cutouts  tc  selected,  terrinals. 


f.  lvalue  tier.  j  u  e  s  t  i  cr.r.a  i  re 

Following  8  study  or  the  ree sures  cf  operational 
effectiveness  which  nay  he  acnieved  ir.  this  interactive 
exrerirent,  an  evaluation  cue  st  icnna  i  re  shod!  be 
constructed  tc  estatiisn  the  level  to  which  these  reesures 
nave  teen  satisfied. 

3 .  Ter.erel  Context 

a.  Concert  and  heed 

A  corplete  technological  esspssrer. :  of  the 
capability  cf  STAf"\N  ZR2  includes  an  evaluation  ir.  a  realistic 
environment  with  a  dynamically  changing  data  tese.  ''he 
exrerirent  rust  subject  STAyMF.2  tc  a  wide  cf  situations 
under  varying  levels  of  sensor  activity  in  ar.  effort  tc 

exercise  all  carabilities  cf  S?AivMR2  and  try  tc  .robe  the 
"limits"  of  those  capabilities.  These  Units  could  be  the 
size  of  data  base,  nurber  cf  reports,  or  f re-.uency  cf 
reports  . 

s  b.  Ueneral  Situation  and  Scenarios 

'*IS  scenarios  will  te  generated  tc  present  a 

variety  of  sensor  reports  and  levels  of  activity  by 
platforms.  Ar.  evaluator  fariliar  witr.  tactical  war  taring 
will  observe  the  play  cf  the  ga-'e  wnile  exercis  in 3TAI-TF?.£ 
tc  the  raxirun  extent  pcssible. 

Zac  a  experimental  sutject  will  =?c  through  a 

short  ST.4PMR2  indoctrination  session  during  which  he  will 
be  given  a  surra ry  cf  the  tnecry  and  i^plp~entat  icr.  cf 
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STaCMRI,  a  threat’  triefir...  to  explain  the  srereric,  -r.d 
a.n  overview  of  what  is  expected  cf  hir. 

luring  the  course  cf  the  actual  ex  peri  rental 
run,  the  evaluator  may  be  assisted  by  the  VIS  operators  end 
test  director  in  the  mechanism  cf  input,  output,  ar.d  display 
to  remove  extraneous  or  distracting  details  of  the  era  me  . 

4  .  Zv  a  1  ua  t  i  c  r. 

a.  lata  Collection 

The  following  data  should  be  collected  for  each 
scenario:  the  tines  at  which  the  date  case  is 
updated,  the  situation  analysis  by  STAt“PIS<,  and  the 
situation  analysis  by  the  operator. 

These  data  will  be  du^nented  by  the  situation 
assessments  ~ade  by  tne  test  director  whc ,  having  become 
familiar  with  the  full  scenario,  can  assess  the  "ground 
truth"  by  having  Knowledge  cf  the  actions  cf  ail  platforms 
throughout  the  scenario.  The  ^ues t ionr.a  1  re  >iven  to 

each  operator  should  consist  of  questions  aesift'ned  to  assess 
tne  real  is-"  cf  the  scenario  and  the  utili  ty  cf 
ST  AW"  I  HZ  as  objectively  as  possible, 
b  .  Ana  ly  s  i  s 

Comparison  is  made  between  1-he  situation 

% 

descriptions  by  STAfc.v.IE2,  the  evaluators,  ar.c  s  round  truth 
to  assess  the  general  accuracy  of  these  descriptions.  The 
questionnaire  results  ray  be  presented  in  tabular  form  and 
tne  objective  comments  discussed  in  the  conclusions . 
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Anticipated  Results 


'.vnile  it  tr ay  be  possible  to  sene  rate  saturation 
pcirts  for  STAKMI-.JL ,  -an i Test  ir  tire  *  el  37s  wnereio  the 
STA*  MR2  display  s  ir:ni  f  i  cantly  lass  behind  the  >:S  scenario, 
the  serial  rat u re  cf  revert  prccessin?  would  see~  to 
trecluie  this  possibility,  providirs  the  query  fvrrticr  cf 
STAl'b'ZR  2  is  not  extensively  usea. 

I .  Cements  and  Special  Ins  tru:ct  iers 

The  scenario  tare  should  be  r.a in tai ned  sc  that  a 
siven  game  ray  be  replicated  exactly  witr.  zero  variance  for 
different  operators  ar.i  so  that  the  test  director  car. 
thoroughly  deterrice  the  situations  as  they  exist  (or  ere 
about  to  exist'.  Kevins  the  sames  cr  tare  vruld  allow  trials 
without  VIS -trained  operators  to  input  the  instructions 
required  by  a  detailed  script.  In  addition,  ar  observer 
could  rrenitor  the  Flue  forces  in  one  or  rore  cf  the 
scenarios  in  a  run  without  STAMfFR2  and  "lake  oc-pariscr.s 
with  the  exact  run  without  ST.-.WIR2.  Finally,  if  the  runs 
were  net  available  on  tape,  very  detailed  scripts  describing 
all  of  the  actions  of  both  the  Flue  ard  Grange  forces  ri'ht 
be  written  for  each  of  the  scenarios  to  accomplish  the 
objectives  of  this  test. 
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TAG  * -THREE  EXPIRIM.' 


1 .  Objectives 

The  sta^e-three  experiment  cas  as  its  objectives  to 
;..rrvibe  an  operational  evaluation  of  the  military  utility  of 
STA.*-MH2  to  a  decision  raker  at  ar.  afloat  task  force  ccrre.nd 
and  control  center  in  a  simulated  environment  and  to 
evaluate  whether  STAMM  R2  improves  the  decision  raker's 
ability  to  rake  rapid  and  accurate  decisions.  As  in  the 
case  of  the  $  tame -one  experiment,  this  level  of 
experimentation  are  discussed  nere  tut  were  not  performed  as 
part  of  this  research. 

2 .  Resources  Renuirei 
a  .  Techni cal 

The  STACIE!  and  MS  pro. -rams  and  associated 

corputer  hardware  are  required  for  this  experiment. 
d.  Display 

An  alrhanuireri''  display  syster  is  required  for 


eaM  tea 

^  c  f 

players 

and 

the  uvpire  tea".  The  G E.\' 

system,  a 

?clo  r 

^rap  hi c 

disc 

lay,  may  be  used  tc  present 

situation 

which 

is  A*ene 

rated 

by  WES  . 

c .  Scenario 

An  opera  tic  r.al  ly  realistic  scenario  is 
f./enereted  to  test  STAMM?!  in  a  controllable  tactical 
situation.  The  scenario  should  include  various 
infcrratior.  tcourse,  speed,  sensor  r«  ports,  i  r.  t*-l  1  iwer.oe  ' 
available  to  a  Tactical  Action  Officer  tc  describe  the 
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sit  ration  as  it  is  norrally  presented  by  'VIS.  vis 
visual  display  and  staius  boards.  The  tears  v»iil  be  allowed 
tc  interact  with  units  under  their  c  errand  by  rrJerir/ 
course  sr.i  speed  changes,  enatlir.,-:  various  sensors,  and 


executing 

tactical  orders 

a,: a  ins  t  an 

opponent.  "he 

urpire  re:, 

insert  various 

elements  of 

i  r  t e  1 1  i  ge r.ce  as 

desired . 

d  . 

Personnel 

The  personnel 

required  for 

the  exper Trent 

include  observer  tears  to  act  as  rlue  and  Orange 
iorces,  a  scenario  controller,  an  venire  tear,  and  a  t *=s t 
director. 

e.  Support  Prceram 

In  addition  to  scenario  genera  ting*  re '\ui  rere  n  t  s  , 
software  tc  r.easure  and  record  tne  tires  at  which  the 
ressages  are  output  and  the  tires  designated  by  the 
observers  is  required.  This  prcgrar,  based  on  the 
APPANET  TV?FSC?IFT  feature,  records  all  irnuts  and 
outputs  to  selected  temr.als. 

f.  Evaluation  questionnaire 

Following-  a  study  cf  the  measures  cf  operational 
effectiveness  which  ray  be  achieved  in  this  interactive 
fxperirent,  an  evaluation  questionnaire  should  be 
constructed  tc  establish  the  level  tc  which  these  reasures 
..ave  t ■  ••  :i  satisfied. 
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Z .  General  Context 

a.  Concept  and  Need 

It  is  possible  that  the  value  of  ST  Ac  M?-2  as  a 
decision  aid  ray  only  be  assess ed  in  an  cjeretional 
scenario.  ?cr  example,  it  ray  enable  a  decision  "‘a her  to 
better  understand  the  threat  situation  which  faces  his 
femes  and  tc  ",ake  ^cre  accurate  and  tirely  decisions. 
These  experiments  would  pit  two  opposing  decision  r a hers 
against  each  ether  in  a  simulated  command  and  control 
environment  and  seek  to  measure  the  value  of  STAPMH2  by 
ccmparir.s  the  decision  rakers'  performance  operating  with 
and  without  STArMl?.2. 

t.  Seneral  Situation  ard  Scenarios 

The  operational  evaluation  of  STAFMF.2  requires 
cpenaticr.ally  realistic  scenarios  consisting  of  Blue  and 
Crer.pe  task  forces,  which  w'uli  be  acted  out  with,  each  tear 
piven  the  flexibility  cf  exercising  complete  control  over 
their  forces  as  len*  as  they  do  not  countermand  their 
ordered  missions.  For  each  scenario,  an  operational  measure 
cf  effectiveness  (bOZ)  or  multiple  HOZs  should  be  selected 
and  used  tc  assess  the  operational  utility  of  $TANfIZ2. 

Because  cf  the  free-play  flexibility  y i v e n  tc 
the  Elue  and  Cran*~e  force  corranders,  the  war  wares  and  the 
resulting  outcomes  may  vary  s i.-nif icantly  from  trial  to 
trial.  Conseq uen tly ,  each  scenario  snouid  be  replicated  in 
such  a  way  as  tc  avoid  f c reknew led <, e  by  a  playing  tee  ~ .  This 
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ca n  be  done  by  usin*  a  nurber  of  teams  '.based  on  the 
availability  cf  assets^,  with  multiple  scenarios,  oy  varyir* 
the  use  ana  non-use  of  STA^Er-Z  by  a  tear,  (considering  that 
a  tear  is  expected  tc  play  ir.cre  than  one  .ya^e ,  a  comparison 
of  pe_rforrcr.ee  based  on  the  b’CEs  ray  be  made. 

Each  tear  should  r c  through  a  brief 
i nice t r i r.a t ic r.  session  during  which  the  following  points 
will  be  discussed:  the  theory  and  i r p  lamentation  of 
ST-AN^EF.2,  a  threat"  briefir?  tc  explain  the  scenario  and  an 
overview  of  the  mission,  ar.d  an  overview  of  the  measures  of 
effectiveness . 

4  .  Eva lua  t ion 

a.  Data  Collection 

The  following  data  should  be  collected  frr 
each  scenario:  times  at  wnich  the  data  base  is 
updated,  situation  assessments  by  STAdTZ?.2,  situation 
assessments  by  the  teams,  actions  taken  by  the  decision 
maker,  ar.J  data  required  tc  determ'  the  selected 
operational  measures  of  effectiveness. 

These  data  nay  ce  augmented  by  the  situation 
assessments  made  by  the  test  director  who,  havin.r  be  cert 
familiar  with  the  full  scenario,  can  assess  the  'mrcund 
truth"  by  bavin.-.-  knowledge  cf  the  actions  of  <=11  platforms 
throughout  the  scenario.  Tie  puestionnai re  riven  tc 
each  operator  should  consist  of  questions  ■’esik-nei  tc  assess 


the  realise  of  the  scenario  arid  the  utility  of 

S1r^M?.2  as  objectively  as  possible. 

b .  Analysis 

Data  summaries  ray  be  trade  for  sacn  scenario  and 
tabular  displays  should  be  rade  cf  tr.e  f-CZs. 

A  correlation  analysis  should  be  rede  with  the 
time  seouer.ce  cf  actions  taken  by  the  decision  raker 
based  on  the  situation  assessments  made  by  STANMR2  to  try 
to  determine  hew  much  the  commanders  utilize 
STAhMP.2  end  what  information  is  most  useful  to  them  t. 
comparison  cf  the  same  outcomes,  as  reflected  by  the 
hOIs,  may  be  made  usine'  analysis  of  variance  to  see  if 
there  is  any  significant  difference  due  to  STA;-T:?S2 . 

A  comparison  may  be  made  of  the  ^amt  outcomes 
and  the  subjective  appraisals  cf  ST AM*  132  to  see  if  there 
was  c  relationship  between  how  well  the  forces  performed  and 
hew  much  they  liked  STAvf/ZR2. 

c.  Anticipated  Results 

It  is  anticipated  t.nat  there  could  be  a  lar=re 
variance  in  the  overall  end-  cf-.vame  operational  measures  cf 
effectiveness.  The  possibility  exists  that  the  vCrs  will  be 
sufficiently  ambiguous  that  any  signal  due  to  S?AtvM'R2  ~ay 
not  be  discernible.  Thus,  the  subjective  evaluations  of  the 
teams  may  provide  tne  only  discrimination  in  the  test,  with 
no  «uar.titative  assessment  of  the  value  of  STA^'MEZ 
possible. 


* 
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Zhe  correlation  analysis  of  the  derision  maker's 
actions  and  the  dTAt-’MEZ  essessrsnt  should  reveal  the  type 
cf  infornatior.  that  is  Test  useful  tc  a  decision  r.a >e r  and 
new  ruch  he  jay  ^row  to  rely  cc  STAHIR2.  Overall,  these 
experiments  should  yield  important  information  acout  the 
a ccer ta bi 1 i t/  of  STAi^MSZ  to  the  decision  raker  and  its 
utility  to  him. 

i .  Comments  and  Special  Instructions 

The  usefulness  cf  ST  Ah  MHZ  tc  a  decision  maker  may 
strongly  depend  on  the  manner  in  which  STAMhlEZ  outputs  are 
displayed  tc  the  Blue  forces.  This  mi-'ht  be  especially  true 
in  hisfc  density  situations  with  a  backlog  of  unprocessed 
ser.scr  reports.  Tne  nature  cf  the  display  dedicated  tc 
ST  At-  KIR  2  should  be  considered  prior  to  ste^e  three 
experirents.  The  use  of  a  terminal  ether  than  the  graphic 
display  terminal  for  auery  and  response  actions  allows  the 
picture  tc  remain  undisturbed  until  refinement  of  scale  or 
sequential  processing  is  requested.  The  Jispiay  syste~  is 
an  integral  part  of  STAK-fdFSZ  so  the  evaluation  should  ce 
based  cn  the  most  flexible  display  available. 

A  tape  record  of  the  &'Srr.e  should  be  made  of  the 
inputs  ana  outputs  of  each  ^ame  so  that  evaluation  personnel 
can  reproduce  a  «pi ver.  trial  and  conduct  a  review  cf  any 
pcint  to  assist  in  "what  if”  types  cf  analysis. 
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The  scenarios  used  should  te  crerat  1  cr.aliy 
realistic,  and  tr.e  starting*  conditions  and  measures  of 
effectiveness  rust  ce  selectee  to  exercise  the  full  ratine  of 
STAf- THP.Z  eapatilities  ar.d  tc  yield  useful  information .  The 
scererios  reed  r.ot  be  developec  Just  for  the  purpose  of 
exercising  STAMVEF.2.  Instead,  tr.e  Hue  a r.d  Orange  task 
forces  should  ref 1 e ct  as  close iy  as  possible,  within  the 
constraints  cf  resources,  the  composition  cf  forces 
currently  available  or  projected  as  desired  and  the  types  of 
rissiens  norrraily  undertaken  by  eaoh. 

The  subjects  used  i r  the  experiments  ray  be  very 
irportant.  For  purposes  cf  scientific  credibility,  the 
subjects  ~ust  nave  command  and  control  experience  at  hi.ph 
levels,  preferably  wavy  Captains  (C-c'  or  Admirals.  A 
potential  criticisms  cf  experiments  which  attempt  tc 
determine  the  operational  utility  of  a  system  is  that  the 
' operators'  aid  not  reflect  the  potential  user  population. 
This  wculd  seem  especially  true  when  the  evaluations  depend 
heavily  on  subjective  appraisals.  There  are  practical 
difficulties  in  cbtainin*;  suer,  test  subjects.  However,  the 
issue  is  whether  tnese  tests  can  be  viewea  as  scientific 
experiments  c  r  demonstrations  without  tr.e  rperaticr.al 
realisr  engendered  by  their  p-reser.ee . 
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lXPEH  IMNT 


INSCRIPTION 

A  sample  expe  ri-ne.rt  was  selected  from  tr.e 
preceding  discussion  tc  illustrate  both  the  capability  of 
STaNM?S2  and  and  t  ne  practical  application  cf  these  methods. 

The  stare-two  experimental  methodology  was  selected  for 
this  thesis.  Selection  criteria  included  the  expertise  cf 
the  test  subjects  available  ana  the  availability  cf  the 
facilities  at  the  Navel  Postgraduate  School.  The  ttst 
subjects  selected  were  military  officers  with  background  in 
ccTar.d  and  control  concepts  rained  through  formal  education 
and  a  wide  range  of  military  experience.  The  Com rand. 
Control,  and  Communications  Laboratory  contains  extensive 
gratnic  and  text  capability,  and  the  STANNNPP  ar.d  V~S 
software  are  available  at  the  Naval  Ccear.  Systems  '"enter 
tNCSC',  San  Diego,  California. 

The  Warfare  Lr.vi  rcr.r'e  r  t  Simulator  (VIS,  war  ra^e 
procedure  involves  the  construction  of  a  computer  file  vnicn 
contains  identification  cf  ail  "esired  players,  initial 
positions,  courses,  ar.d  speeds.  Flatforrs  may  he  selected, 
free  a  resident  data  base  vhicn  assigns  sensor  and 
weapons  configurations  tc  the  units.  rcl  levin.:  r*are 
initialization,  the  u~its  may  be  -’areuvered  cr  cc— ar.d  from 
the  player  through  a  computer  terminal,  with  display 
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of  exact  .feorfrat  .ii c  location  of  mnc*".  units  available  on 
selected  -orarhio  display  reciters.  Ir.teractier  ef  tr.e 
i  cits  ray  be  ordered  by  the  players  (as  in  attacks),  but 
ser.scr  reports  are  dependent  c ”  detection  al.-Tritr.^s  ir. 
>. IS .  For  example,  detections  by  a  surface  search 
radar  against  another  surface  vessel  occurs  at  2?  rile 
minimum  separation.  Ser.sor  reports  are  =.iven  to  the  player 
via  a  status  hoard  text  display  and  on  the  yec^raphic 

display  which  is  updated  every  =jare  mir.ute. 

This  experiment  was  net  interactive  in  that  the 

eperat cr  did  net  influence  the  s er i es  c f  events  which 

were  heir*?  generated  by  IVFS.  It  was,  however,  interactive 
in  tr.e  sense  that  the  operator  nad  tr.e  capability  tr  select 
different  status  board  displays  and  change  ranee  scales 
as  desirea.  The  fully  interactive  war  care  as  suwmested  in 
the  third  level  ef  experiment  wcvld  allow  operator 
influence  of  the  event  stream.  The  prepare-5  scar.aric 
used  in  this  experiner.t,  waion  included  initial 
positions,  initial  courses  end  speeds,  arm  orders  to 
the  various  units  are  included  in  Appendix  F. 

The  scenario  for  this  experiment,  venerated  in 

a  IS ,  consisted  of  three  U.5.  warships  oc  rdu  ct  i  a 

surface  surveillance  of  a  merchant  lane  in  the  vicinity  of 
the  forth  Atlantic,  vitn  neutral  ari  • pc te- t iai  ly  !  hostile 
platforms  in  the  area.  The  warships,  assisted  by  patrol 
aircraft  ccndvctinr  similar  surveillance,  were  to  locate, 
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trac’x,  a  r.d  identify  ct.uer  units  in  the  area.  "he 
experimental  subject,  embarked  on  tne  cruiser  Belknap  ,  .icd 
STA^'EP.Z  available  to  assist  in  that  mission.  The  STA.f- [■''EEf 
stertup  procedure  is  given  in  Appendix  A. 

The  scenario  was  chosen  based  or.  the  ran  me  of  situations 
available  and  the  relative  realism  of  a  nc  r.-eom'cat  i ve 
confrontation.  The  scenario,  tactics,  sens:rc,  a-i  sensor 
c.iaracter is  t ics  are  purely  hypothetical .  The  uata  base 
rAppend  ix  E]  and  rules  ^.Appendix  C]  are  assurptiers  tailored 
to  this  scenario,  cut  are  easily  translatable  to  those  vhich 
a  Tactical  Action  Officer  would  reccmr.  ize  as  havir.m 
potential  applications.  The  briefing  cf  the  subjects 
included  an  explanation  cf  the  artificialities. 

A  difficulty  in  the  actual  conduct  of  the  experiment 
existed  due  tc  tne  nature  cf  the  reports  on  which  STA’-mr?.£ 
operates.  The  sensor  reports  received  by  an  operator  cf  '*10 
are  both  visual  cues  cr  a  mrapnics  display  'lines  cf  tearing 
cr  r.ev  platform  symbolomy),  ana  in  a  table  of  ..detections 
< the  Electronic  Warfare  Status  Beard,  the  Surface  Status 
Board,  or  the  Air  Status  Board).  The  form  of  the  tabular 
report  is  formatted  as  tc  its  numerical  secuer.ce. 
Her  t  If  Ice  t  icn  t  end  location  *  i  f  applicable,.  The  reports  as 
generated  by  i»IS  are  not  readable  by  STAMMS?.  Cas  example 
cf  the  disparity  is  that  STAGES!  recognizes  latitude  and 
lonmitude  in  degrees  and  tenths  rather  than  degrees  and 
-irutes.  The  LISP  version  cf  »£S,  L>  ES  ,  dees  report  in  the 


w  C 


c  t.  j  r  c  c  r  i  c  1 6  forget,  z '•  *  t  t  h  p  difficulty  i  r  v  5  i  r. ,  *  1 y  z  f  c  e  h 
as  ccrpiex  scenario  veneration  and  limited  output 
capability,  far  cutwei.’h  this  advantage. 

The  solution  to  tie  for  rat  disparity  lies  ir.  the 
nature  of  ’.vZS.  »ith  duplicate  p  la  tfor^s ,  starting  at  exact 
locations,  given  precisely  the  same  order  cf  instructions 
i such  as  course  changes  cr  sensor  status',  sense r  ; sports 
are  also  duplicated.  That  is>.  the  detections  are  based  cn 
raxirur  ranges  of  detection  which  are  unchanged  and 
vnich  are  combined  in  the  *IS  detection  algorithms  ir.  the 
sare  way,  to  produce  duplicate  reports  at  the  sere 
_-a~e  t  ire.  This  characteristic  was  exploited  by  rrnninv  a 
pare  to  a  suitable  pcint  ir.  tirre  usir.v  the  AT.PaMT 
typescript  feature  which  copies  ail  ter~in=i. 
interactions  as  they  occur.  This  typescript  shows  the 
various  reports  received  during  a  gare,  complete  with  the 
tire  cf  receipt.  Syr.thssi  sing  ga^e  reports,  a  complete 
file  of  reports  [Appencix  L j  was  built,  in  the  ferrat 
fre"  which  STAFMSRZ  could  read  its  input  ^ata.  Therefore, 


STAbMP.2  was 

net  receiving 

di rec t 

cv tpu t  f  rcr 

the  sensors, 

but  via  an 

intermediate 

process 

net  part  cf 

€?  1  t  ’  1  &  !*  W  ~  ^  c  " 

STAbMEZ. 

STAm?.2,  using  the  Tit-TRCNIX  4214-1  terminal  for  beth 
text  and  graphic  display,  vas,  in  effect,  a  separate  entity 
running  concurrently  w i t n  the  gar e .  The  cor p arisen  of  the 
subject's  perspective  of  the  situation  and  that  cf  STANVF^Z 
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is  still  valid  only  as  lory  as  s  ir  ul  tanecus  reporting  is 
carefully  i n t a  i r. e J  . 

The  art  if icial i ty  of  the  message  input  could  neve  biased 


the 

experiment 

to  a  rerar  ■cable 

j^h^CO 

hu 

t  STANUERZ  has 

o  r.e 

feature  which  ameliorates  this 

..rc  bl  er 

* 

STANMR2.  do^s 

not 

process 

the  next  sequential 

report 

in 

the  file  until 

the 

user  q u i t  s 

the  query  rode  of 

c  p  e.  r  a  t  i 

on 

'hat  is, 

uati 

1  the 

quit"  c err and  is 

-q  i  v  e  n 

there  is  no 

processing  of  the  next  sequential  report.  Recause  the 
experimental  subject  is  r.ct  expected  to  be  the  actual 
operator  of  the  STr.yM?2  hardware,  the  operator  car,  he 
instructed  not  to  leave  tne  ouery  mode  until  the  appropriate 
^are  tire  (which  is  displayed  on  The  input/output  terminal 
for  ’*ZS  ) . 

icllcwing  a  suitable  period  rare  tire  where  the 
subject  is  encouraged  to  exercise  STAh-YSF.2  a  questionnaire 
was  presented  to  gather  the  impressions  and  appraisal  of  the 
experiment  in  general  aac  S?Ari\ZR2  in  particular. 

£.  RESULTS  AN L  CONCLUSIONS 

The  questionnaire  supplied  to  test  subjects,  with  a 
suirary  of  results,  is  listed  below.  The  intent  of  the 
questions  was  to  verify  the  "realism"  of  the  scenario  ard 
to  gather  a  subjective  evaluation  cf  STACKERS.  The  scoring 
scale  of  1  to  10  was  arbitrarily  chosen,  with  10  the 
cost  positive  or  rost  favorable  response.  The  test 
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sutjects  consisted  cf  eight  i\a?y  and  Air  Force  officers  , 
ranging  in  brede  from  C-3  to  who  are  members  cf 

the  Command,  Control  and  Ccmrcur.  icat  icr.s  curri  c  ulum  at  the 
A  aval  Postgraduate  Schccl*  f^onterey*  California*  A.  11  have 
beer  exDcsed  tc  decision  theory  ari  to  an  overview  cf  the 
revel  tactical  environment.  The  Navy  participants  included 
tnree  C-4s  and  cne  0-3,  All  but  ere  e*  the  havy  0-4S  had 
Tactical  Action  Officer  training* 

STA^EEZ  QuISTIONwAIHI 


1.  Prior  to  the  scenario  presentation,  was  the  purpose  of 
STAMPERS  adequately  explained?' 

12340670910 

Responses  1132  1 

Average  6.125 


2.  Was  the  scenario  representative  cf  your  previous 
experience  with  WPS  or  L>ES? 

1234  5  €709  1C 


Responses  1 

Average  6.375 

3.  Was  the  scenario  realistic? 

1  2  3  4  5 

Responses  1 

Average  6.373 
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9  9 

1  3 
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4 .  a  e  r  e 

t:ie  displays  consistent  with 

t  he 

5  e  r  s  c  t* 

repcrts  ? 

12  2-55 

r* 

r 

u  12 

Re  s  p  c  r.  s  e  s 

2 

2 

2  1 

Ave  ras.6 

?  .375 

5  .  V'as 

sufficient  tire  available  to 

understanc 

the  senso 

reports  ? 

12  3  4  5  5 

7 

9 

9  1  i 

Responses 

1  1  1 

2 

1 

2  1 

Average 

7.5 

5 .  were 

the  STAMMR2  graphics  clear? 

12  2^55 

7 

g 

S  1 2 

Re sp cases 

1  13  2 

1 

Average 

4 . 5 

7  .  Was 

the  S  TAMMS  £  commentary  understand 

able? 

1  2  2  4  5  6 

»7 

6 

9  lr 

Res  por.s  es 

1  3 

1 

2 

1 

Average 

6.575 

5  .  a  e  r  e 

the  STA.VMR2  conclusions 

ccn  s 

i  5  ten  t 

wi th  ycv 

conclusions? 

1  2  2  4  5  5 

7 

6 

9  12 

Responses 

C 

-z 

1 

Average 

7.25 

3.  "id 

you  use  the  explar.at  icr.  trace? 

1  2  2  4  5  5 

n 

5 

9  12 

Responses 

1 

*T 

3  1 

^vere^e  ? .275 
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12.  If  the  exrlar.ati.cn  trace  vas  used,  v ere  the  assertions 
consistent  with  the  conclusions? 

122455729  1? 

Responses  2312 

Average  7.275 

11.  >as  the  presence  of  STAf“MR2  distracting? 

12341  5  7  E  9  1  £ 

Responses  13  2  11 

Average  4.75 

12.  *ould  an  assistant  who  would  filter  inforretior.  fror 
STANh2?.2  to  the  rinirutr  essential  level  be  useful? 

1234  he  72912 

Responses  11  32 

Average  &  •  5 

13.  Is  ST AI*VI?2  useful? 

123455729  17 

Responses  1  142 

Average  7.75 

The  results  show  an  overall  trend  towards  an  average 
response  which  could  be  considered  favorable,  in  the 
vicinity  of  6.  The  exceptions  provide  a  clear  contrast 
consistent  with  conditions  noticed  during  the  experirer.t. 


Question  5  had 

the 

lowest  average  ran 

ici  :ig . 

The 

difficulty  encountered 

with  the 

display  ray  beet 

be 

described  by  shoving 

the 

display  at 

three  sel 

ected  ear? 

"i r.utes  .  Figure  1 

is 

the  last 

d lspl ay 

without 

bearing  lines,  which 

o  ecu 

rs  at  rinute 

2.  Figure 

2  i  s 

the 

disrlay  at  'are  nirute  14.  vr.icr.  includes  intelligence  data 
(MP.l  ar.4  MP2),  surface  contact  reports,  ~.'f  bearing  licet 
and  t ne  location  cf  friendly  units.  Figure  T  is  tne  display 
at  gare  minute  4d  end  is  a  compendium  of  all  reports 
received  to  that  point.  Close  observation  of  the  screen  as 
the  graphics  were  being  drawn  shewed  a  sequential  entering 
cf  the  information  as  it  was  being  drawn.  Sore  reasure  of 
order  was  discernible  if  t:.ls  process  was  carefully 
followed,  aowever  the  display  is  unreadable  after  it  is 
c eyeleted  . 

A  modification  to  the  display  pecica^e  is  required.  Cr.e 
suggestion  fror  the  test  subjects  was  that  only  the  last  two 
occurrences  cf  a  contact  be  displayed,  with  earlier  reports 
available  for  historical  review  as  directed  by  function  hey 
s  el ect i or  . 

A  benefit  of  STAIPldl?*  was  demonstrated  during  the 
experiment.  In  every  iteration  of  the  display,  intelligence 
information  in  the  form  of  contact  reports  received  from 
external  sources  was  present  cn  the  screen  as  a  reminder 
tc  the  decision  raAer.  Because  a  suspension  rcr.cernir-r 
this  information  is  created  ir.  memory,  this  information  was 
available  at  all  ti^es  for  review  arc  explanation  as 
desired.  This  information  was  not  subject  to  update  in  this 
scenario  and  did  net  seem  to  contribute  to  tne  general 
clutter  discussed  above. 

££ 
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Jit-  2  STAM^iRc  Graphics  display,  Gere  r- '  i  n  ^  14 
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There  appeared  to  be  a  disparity  between  the  STALER? 
conclusions  and  those  cf  the  test  subjects  (questions  t  a  r.a 
IT;.  The  difficulty  was  .Test  apparent  in  cases  where  the 
test  subjects  were  able  to  cross-fix  passive  detection 
tearing  lines  tc  a  geographic  position.  Rather  than  conclude 
that  an  emitting  platform  exists  at  that  point  cf 
intersection,  STAMPERS  would  continue  tc  make  conclusions 
based  only  on  the  individual  ?V  reports.  Here  the  raj  or 
difficulty  appeared  tc  be  in  the  construction  cf  STANPIR2 
rules  rather  than  the  functioning  of  the  rules  that  exist. 
A  rule  correlating  or  position-fixing  multiple  passive 
detections  of  the  same  emitter  would  appear  to  be  a 
relatively  simple  matter. 

The  explanation  trace  feature  cf  STaN^FF.S  proved  to  be 
valuable  as  both  a  decision  aid  and  as  a  trailing  tool, 
while  the  users  commented  (question  9)  that  the  trace 
included  extraneous  inf  crma.tion  ,  the  level  cf  ucd°rsta  nd  i ng 
cf  the  scenario  increased  markedly  due  to  The  increased 
appreciation  cf  what  rules  were  beinir  apt  lied  ana  hew 
STAhMRc  applied  them. 

The  lew  average  score  for  question  11  does  not  fit 
the  general  pattern  cf  the  cues t icnr.ai ne  due  tc  the 
wording  of  the  question  .  The  negative  response  indicate? 
that  STAKMR2  was  not  distracting  to  the  user.  The  potential 
exists  for  the  test  subjects  to  became  engrossed  i n  the 
execution  cf  STAyvEA2  and  the  explanation  trace  tc  the  point 


where  the  conduct  cf  the  game  decodes  a  minor  consideration. 
This  apparently  was  r.ot  the  case  here. 

The  rereinder  of  the  ave ra6es  appear  to  be  generally 
consistent  with  favcraole  response  to  both  the  scenario  in 
general  and  to  STAMdIB2  in  particular. 

This  experiment  depends  heavily  or  subjective 
evaluation.  While  the  scenario  and  operation  of  STAVPE52 
appear  to  have  been  convincing  to  the  test  subjects,  there 
are  no  clear  measures  cf  operational  effectiveness  which  car. 
be  quantified  or  analyzed.  Given  that  the  computer  capacity 
is  available  to  the  location  of  the  decision  raker,  STAWIEZ 
would  appear  tc  be  a  welcome  addition  to  the  repertoire  cf 
decision  aids  available  to  a  commander.  While  this 
addresses  the  original  objective  namely  the 
demonstration  and  evaluation  cf  STAPKEE2,  it  cannot 
full/  answer  the  question  of  overall  utility  of  the  system 
to  a  decision  nauer  ir.  a  real  military  environment. 

Due  to  the  experimental  nature  of  STAKES?.?,  operator 
action  was  required  for  the  message  input  procedure. 
The  requirement  tc  leave  the  explanation  trace  before  the 
next  message  can  be  read  is  purely  artificial  and  er. 
accepted  penalty  in  order  to  run  the  program  simultaneously 
with  WES.  In  an  operational  system,  the  mechanism  for  the 
transfer  of  data  between  a  sensor  ana  the  decision  maker 
should  be  understood  and  automatic.  The  addition  of 
procedural  steps  such  as  function  key  or  tfrminal  inputs  in 
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crier  t  c  receive  a  sens  or  report  ta  y  te*-d  t  r  te 
ignored  during  tires  of  stress.  In  addition,  this  an  to  ratio 
transfer  ^ust  te  regulated  tc  a  suitable  tire  period  between 
reports.  Reports  from  multiple  sensors  at  short  intervals 
would  rapidly  overload  an  operator  required  tc  view  every 
report  . 

STAN'MIRS,  as  an  i  nves  t  iga  t  i  on  into  the  utility  of 
artificial  intelligence  as  applied  to  corn, and  and  control, 
has  progressed  beyond  the  concept  and  technology 
deronstra t ion  phase  of  its  existence.  The  somewhat 
artificial  nature  of  the  message  transfer  procedures  is 
capable  of  approaching  a  "real  world'  condition  in  tnat 
machine  and  human  readable  formatted  messa?es  are  ir.  common 
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Pacific'',  could  further  refine  them  tc  reflect  mere 
realistic  decision  conditions. 

A  decision  aid,  even  one  directed  toward  a  unique 
user  ve.g.,  the  Tactical  Action  Officer),  may  need  tc  te 
distributed  througnout  a  command  center.  The  mass  cf  data 
presented  may  require  the  decentralization  cf  control  tc  the 
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point  where  the  iAC  cr  staff  watch  officer  function  re;  be 
re legatee  to  relatively  discrete  areas  of 
responsibility,  ’a h i  1  e  the  requirement  for  a  prime  decision 
maker  exists,  this  role  may  well  ce  one  of  command  by 
negation.  There  is  a  tire  penalty  for  reevaluating 
previously  screened  information  ir.  a  high  density 
environment.  Fach  decision  maker  would  then  require  his  own 
version  cf  STAMMS 2  that  contains  a  tailored  rule  set. 
The  present  implementation  of  STAMMP2  already  contains 
rules  which  may  be  useful  and  trers porta  ole  to  this 
practical  application. 

The  benefit  of  this  program  atcarJ  a  ship,  cr  at  a  major 
Fleet  command  center,  for  example,  could  be  that  a  "super 
agency",  consisting  cf  intelligence  and  warfare  specialists 
could  provide  a  realistic  assessment  of  situations  which 
right  be  encountered  and  tailcr  propositions  to  reflect 
standardized  intelligence,  doctrine,  and  tactics  in  a  pre- 
loaded  database  ana  system  that  would,  in  real  tire,  be  an 
"oracle”  which  would  provide  assistance  and  guidance  tc  the 
decision  maker.  In  case  of  questionable  data  the  review 
capability  of  the  process  by  which  a  decision  was  reached 
would  assist  the  decision  maker  in  logically  analyzing  his 
own  thougnt  process.  The  training  benefit  cf  the 
preposition  review  could  increase  the  competence  cf  the  TAC 
who  could  see  hew  the  'panel  of  experts'  approaches,  the 
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I r.  summery ,  this  thesis  ha s  presented  beckgrov  ri 
iafcrratior.  concertis?  artificial  intelligence  ar.d 

S?Af-M?.2.  Various  testine  methods  were  presented  tc 
evaluate  the  usefulness  of  STiiPMLBZ  and  an  experiment  was 
conducted  and  examined  which  used  one  of  these  methods. 
STA^MIr.2  appears  tc  be  a  useful  decision  aid  concept  with 
?reat  potential  for  further  research  and  testis,?. 
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APPENDIX  A 

STANDEE 2.  GUILE 

The  following  is  a  rrcceaurai  guide  to  beginning 
tne  STAIhMSP.2  program  for  this  experiment.  After  legging  i-.tc 
the  Naval  Post  graduate  School  Command,  Control, 

and  CoTTuri cat  ions  Laboratory  UNIX  system  ;  EEC  FI? 
11/72),  a  TELNET  link  is  established  to  the  T0PS2?  system 
at  Naval  Ocean  Systems  Center,  San  Liege,  California,  under 
account  name  NPS2.  Following  the  prompt,  the  entries 
with  asterisks  are  entered.  Carriage  returns  '<cr>)  are 
only  required  as  indicated. 


Cc  stammerA  <cr> 

_L0AD(  TIM  PEAK  .CCM  * 

compiled  on  24-Feb-&l  12:29:25 
FILE  CREATEL  24-reb-Sl  12:Zt:5b 
(LISFLCE  redefined) 

(IN-LANS  redefined) 

RESULTPRINTER  redefined, 

TEMPEAKCCKS 
(SENSCRANGE  reset) 

<NPS2>T2MPFAK.C0M.l 

_(SETC  ECMESEIP  (SETC  CWNSEIP  '3ILINAP))  - 

(CWNSEI?  reset) 

\ EOKZSH IP  reset) 

EELKNAP 

jSTAtfhIR)  * 

Welcome  to  version  Z.b  of  the  STAMPER  TSA  system, 
hemory  file?  ^Default  is  l*EvORY.):  N’Ef-.JE  <cr>  * 
fernery  initialized. 

P.ulefile?  (Default  is  RULES  .):  RULES  .  Ji  * 

Pules  loaded 

What  file  would  you  lixe  to  take  messages  frer? 
(Default  is  SCENE. ICS;:  SCENE. JE  <rr> 

Are  you  running  on  a  Tektronix?no  * 

Dc  you  nave  a  Tektronix  available  for  display?  nc  * 
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Passive  detection,  heard  SFS4C  at  bearing  15b. 19  Tire: 
Associated  with  track  E1n.iFI.Z1 

P.ZFOP.T :  3ZRKILEY  was  sighted  ir.  the  merchant  lar.e  LAN 
Cuestion?  Cui t 
Leaving  EXPLAIN 

Passive  detection.  Heard  SPS39  at  bearing  15E.L9  Time: 
Associated  with  track  BERKELEY 

REPORT:  BERKELEY  was  sighted  ir.  the  merchant  lar.e  LAN 
Cuestion?  EF.IAK 
(Explain  broken) 

:dribbie] 
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APPENDIX 


DATA  BASS 

This  is  the  data  base  frcrr  which  STANMISZ  will  draw 
its  first  assertions.  It  has  been  specifically  tailored  tc 
the  experiment  scenario  anc  dees  act  reflect  real- 
world  conditions.  STAN.NER2  will  undate  its  data  base  as 
ressaees  are  received. 


i  MR  CHANT  LANE  LANE1 ) 

l LOCATION  LANll  ((52.20  -22.1)  (c£. 9  -2?.2£))) 
(C»NS HIP  2EIKNAP) 

>•  ID  BELKNAP  FRIEND) 

( ID-ANPLIFT  BELKNAP  ML-BATTLE) 

(ID  BERKELEY  FRIEND) 
i ID-AhPLIf Y  BERKELEY  NIL-BATTLE ) 

(ID  KNCX  FRIEND) 

( 1D-AN.PLI  F  Y  KNCX  NIL-BATTLE ) 

(ID  U201  HOSTILE) 

( ID-ANPLIFY  U021  NIL-BATTLE ) 

(ID  $004  FRIEND) 

;  I D-ANPLIFY  S0£'4  NIL-AUXIL) 

(ID  S005  FRIEND) 

( ID-ANPLIFY  See5  NIL-AUXIL) 
v ID  NER1  FRIEND) 

( ID-ANPLIFY  NFR1  NIL-AUXIL) 

(ID  MER1  FRIEND) 

( ID-ANPLIFY  MSR1  NIL-AUXIL, 
l ID  NER1  FRIEND) 

(ID-AN.PLIFT  NER1  MI-AUXIL) 
tlD  MERE  FRIEND) 

(ID-ANPLIFY  MFH2  NIL-AUXIL) 

(ID  NEP.3  FRIEND) 
v ID-ANPLIFY  MER2  NIL-AUXIL/ 

(id  sees  HOSTILE) 

( ID -AMPLIFY  sees  NIL-BATTLE  ) 

(ID  seee  hostile) 

( ID-ANPLIFY  seee  NIL-BATTLE) 

(ID  see?  HOSTILE) 

( ID-ANFLIFY  S0£7  NIL-BATTLE) 

(ID  HCST4  HOSTILE) 
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APPENDIX  C 


RULES 


The  following  are  tne  production  rules  upon  '"hich 
STAFFERS  operates.  As  lata  frcTi  the  data  tase  or  Tessa?es 
satisfy  the  conditions  in  each  of  the  rules  (labelled 
< ITEM/ )  ,  the  data  stream  is  built.  When  sufficient  data 
exists,  the  rules  fire  and  the  appropriate  actions  are 
carried  out.  The  plain  text  FR IN FC Rti  statement  is  the  rran- 
readable  explanation  of  the  assertion  which  has  been  built 
in  STAMPED. 


INHERIT 

(CONDITIONS  ((ALIAS  *PLAT  *UNXNCWN) 

(TYPE  *PL  AT  *TYP) 

UD  *FLAT  *111) 

( ID-AMPLI EY  *PLAT  *IDMp) 

(CLASS  *PLAT  *CLS) 

(MEDIUM  *PLAT  *MZD ) 

ACTIONS 

((TYPE  ’■'UNKNOWN  *TT?) 

(ID  ^UNKNOWN  *ID1) 

( ID-AMPLI EY  *UN KNOWN  *1DMP ) 

(CLASS  ^UNKNOWN  *CLS) 
l  MEDIUM  *  UNKNOWN  *N!ED  )  } 

CON?  1.3  PRINECRM 

'If  an  unknown  is  identified,  it  inherits  the  properties  cf 
its  identification.”) 

KNOWN-PLAT 

i  CONDITION'S  ( (SIGHTING  *?  *S  , 

(^UNLESS*  (CWNS3IP  *P '» ) 

(FULLY-KNOWN  *P) 

(id- amplify  *p  *i; ; 

ACTIONS 

( ( IDENTIFIED  *?) ) 

CONE  1.0  PP.INFOPM 
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>ark  as  identified  if  known  vita  certainty") 

NCT-IAST-S IGFT ING-V ER2 
1  CONDITIONS  ( (SIGHTING  *PLA?  -Si) 

(PREDECESSOR  *51  *S2) 

( *NOT*  ISAKE-AS  *S2  ML)} 

(♦UNLESS*  (NCI-LAST  *S2))> 

ACTIONS 

( l NOT-LAST  *S  2 ) ) 

CCNF  1.0  PRINFCRK 

If  a  sighting  has  a  predecessor,  tnen  that  predecessor  is 
not  the  last  sighting.’*) 

NCT-FIRST -SIGHTING 
• CONDIT IONS  ( ( S IGRTING  *PLAT  *S  1 ) 

(♦UNLESS*  (NCT-FIRST  *S1)) 

(PREDECESSOR  *S 1  *S2) 

(*NOT*  *  S  Af  E-AS  *S  2  NIL))) 

ACTIONS 

( (NOT-FIRST  *S1)  ) 

CONF  1.0  PRINfORN: 

"if  an  earlier  sighting  occurs  ,  _record  that  the  previous 
sighting  is  not  the  first  sighting."/ 

LAST-VIEW 

(CONDITIONS  ((SIGHTING  *PIAT  *S1) 

(♦UNLESS*  (NOT-LAST  *S1))) 

ACTIONS 

((LAST-SIGHTING  *?LAT  *S 1  )  ) 

CONF  .99  PRINFORF1 

"If  the  sighting  is  not  followed,  it  is  the  last  sighting. 
1.99/") 

FIRST-VIEW 

(CONDITIONS  ((SIGHTING  *PLAT  *S1) 

(♦UNLESS*  (NOT-FIRST  *S 1 )  )  ) 

ACTIONS 

('(FIRST-SIGHTING  *PLAT  *S1)) 

CON?  .(*9  FP.INFORM 

"If  the  sighting  is  not  preceaea,  it  is  the  first  sighting. 

( .99;") 

SI^PLY-RFACHAELE 
(CONDITIONS  ((CONTACT  *C0NT) 

(FIRST-SIGHTING  *CCnT  *S 1  ) 

(SIGHTING  *PLaT  *S2  ) 

( ID-AMPLI  FT  *PL  AT  ML-iATTLZ  ) 
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(•NOT*  ( S  APS  “AS  *CCNT  *?LAT)) 
l •UNLESS*  <  C>' N SHIP  *PLAT ) ) 

(POSITION  *S1  *P1) 

^POSITION  *S2  *P2 , 

(Tos  *si  *n: 

(TOS  *S2  *T2) 

( S’iR  *?i  *T1  *P2  *12) ; 

ACTIONS 

(  (SINPLY-WITEIN-RZACH  *S1  *S2)) 

CONF  .96  PRINFORM 

If  a  contact's  sighting  could  travel  to  a  * IL-BATTLE's 
sighting,  then  they  are  simply  reachable.  (.98)’’) 


REACEA5LE 

(CONDITIONS  ((CON’TaCT  *CONT) 

(SIGHTING  *CGNT  *S1) 

(SIGHTING  *PLAT  *S2) 

(•NOT*  (SAME-AS  *PLAT  *CCNT)) 
(*UNLZSS*  ( CkNSHIP  *PLAT ) ) 
(SIMPLY-W I  THIN-PEACH  *S1  *S2) 
(*UNLESS*  (BLCCKED-FROM  *S1  *52))) 
ACTIONS 

(  C*ITEIN-R2ACE  *S1  *S2  '  ) 

CONF  .97  PP.INFCRM 


If  twc  sightings  are  vithir.  reaca  cf  each  ether,  and  are 
not  blocked  by  patrol  overflights,  then  are  reachable  from 
each  other .  ( .97 )  ’  ) 

COULD-BF-C CM SATAN T 
(CONDITIONS  ((CONTaCT  *COM) 

(FIRST-SIGHTING  *CONT  *Sl) 

( ID- AMPLIFY  *PLAT  ML-BATTL2) 

(•UNLESS*  ( CVNSEIP  *PLA T  )  ) 

(LAST-SIGHTING  *?LAT  *S2) 

( WITH  IN -PEACE  *S1  *S2  ) } 

ACTIONS 

(  ( KNO(»N-CCMB  ATANT  *CONT  )  } 

CONF  .15  PRI.NiORM 

’If  a  contact's  position  could  be  reached  by  a  knevn 
combatant,  then  the  contact  might  be  a  ccmbatant  (.15).”) 

i.OT-KNCVN-COMEATANT 
(CONDITIONS  ((CONTACT  *CONT) 

(•UNLESS*  ( KNOWN -COMBATANT  ♦CCNT))) 

ACTIONS 

( (TYPE  *CCNT  MERCHANT  )  ) 

CONF  .45  PR INFORM 
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If  a  contact  could  not  be  any  known  combatant  as 
ce termi neo  by  rule  CCULD-B E-CCMBATANT ) ,  then  it  may  be  a 
merchant  ( .45) .  , 

FOSS-FPT 

(CONDITIONS  ((PATROL  *FTL ) 

v  CONTACT  *C0NT; 

(3IGH1ING  *CCNT  *Sl) 

(SIGHTING  *?LAT  *S2  ) 
l  ID-AhPLI FT  *PLAT  MIL-BATTLE ) 

(♦UNLESS*  ( CVNSHIP  *PLAT ) ) 

(SOURCE  *S2  *?TL) 
v*NCT*  ( SAME -AS  *S1  *S2}) 

(♦UNLESS*  (DISSIMILAR  *CCNT  *PLAT) ) ) 

ACTIONS 

('.POSSIBLE-REPORT  *CONT  *?TL)) 

CONE  .55  PR  INFORM 

If  a  patrcl  sights  a  MIL-BATTLE  platform,  and  a  contact  is 
similar  to  the  platform,  then  the  patrol  report  concerns  the 
contact .  ( .35 ) ’ } 

BLOCKER 

(CONDITIONS  ( (CONTACT  *CCNT) 

(SIGHTING  *CCNT  *Sl) 
v SIGHTING  *PLAT  *S2) 

(ID-AIMPLIFT  *PLAT  ML-iATTlE) 

(*NCT*  (SAKS- AS  *CCNT  *PLAT ) ) 

(♦UNLESS*  ( OWN SHIP  *PLAT ) } 

(PATRCL  *?TL ) 

l *UNL2SS*  (POSSIBLE-REPORT  *CONT  *PTL); 
(SIGHTING  *PTL  *S3) 

(NCT-LAST  *S3 ) 

(SUCCESSOR  *S3  *S4) 

(POSITION  *S1  *Pi; 

(POSITION  *S2  *P2) 

(POSITION  *S3  *?3y' 

(POSITION  *S4  *P4 , 

(ICS  *S1  *Tl ) 

( TCS  *S2  *T2 ) 

(TOS  *S3  *T3 ) 

(TOS  *S4  *T4) 

( *cr*  ( cross  paths  *pi  *P2  *P3  *P4) 

(GRAZE  *P1  *P2  *P3  *P4) ) 

( ^NQT*  UINT-BIFORE  *P1  *T1  *P2  *T2  *F3  *T3  *pa. 
*T4;  ) 

(*MOT*  (*  ENT-AFTER  *P1  *T1  *F2.  *T2  *F2  *T3  *?4 

*T4 ) )  ) 

ACTIONS 

( (BLOCKED-? ROM  *S1  *S2 ) ) 

CON?  .3  PP INFORM 
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If  a  path  between  two  sis?htinft-  has  not  been  detected  by  a 
patrol,  and  it  would  have  if  tney  were  si*htin*s  cf  tr.e  same 
vessel,  then  they  are  different  vessels.  (.9)"} 

CREATEDETZCT 

■  cor-:i; ii io^’s  ((sighting  *?lat  *sgt) 

(♦UNLESS*  (IDENTIFIED  *PLAT  ) ) 

(♦UNLESS*  ( DETECTION  -FLAT)) 

(SOURCE  *SGT  EW) ) 

ACTIONS 

((EJECTION  *PLAT ) ) 

CONF  l.e  PRINFORM 

"if  the  source  of  a  sighting  is  EW ,  then  mark  it  detected.") 
CRZATECONTACT 

( CONDITIONS  ((SIGHTING  *FLAT  *SGT) 

(♦UNLESS*  ( IDENTIFIED  *PLAT  ) ) 

(♦UNLESS*  v CONTACT  *FLAT  ) ) 

(SOURCE  *SGT  RADAR)  ) 

ACTIONS 

( v  CONTACT  *PLAT) ) 

CONE  1.0  PRINFCRM 

If  radar  is  the  source  cf  a  platform 's  si2htir.£,  then  the 
platform  is  a  contact.) 

CREATE PLAT 

> CONDITIONS  ((SIGHTING  *?LAT  *SGT) 

(♦UNLESS*  ( CVNSEIF  *PLAT  ) ) 

(♦UNLESS*  (PLATFORM  *?LAT  )  ) ) 

ACTIONS 

( (PLAT PORK  *PLAT ) ) 

CONE  1.2  PHI N FCSM 

"Every  si^htinf;  is  a  platform,  except  the  ownship.") 


SMALL-CR AFT9 

'.CONDITIONS  (v  CONTACT  *WEO; 

(FIRST-SIGHTING  *WEC  #S1) 

(SOURCE  *S1  RADAR; 

(RANGE  *S1  ♦El / 

(LESS-THAN  *R1  c) 

(STRENGTH  *S 1  STRONG)  ) 

ACTIONS 

((TYPE  *WHC  SUE) 

(MODE  *WHO  SURFACE)) 

CONF  .Z  PR  INF  CRM 

"if  the  ran*e  of  c  strong  radar  si^htln*  is  less  tnar.  0  nr, 
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and  it  is  the  first  si.eatir.e,  ther.  the  contact  is  possibly  a 
su rf c ce  sub .  (  . 5  )  ' 

Sf'iAI  I  “CRAFT  5 

wccr:iTiONS  '.(contact  *x, 

(SIGHTING  *A  ’‘‘SIGHT  ) 

(NOT-flRST  *SIGHT) 

( RANGE  ♦SIGHT"  *R ) 

( LZSS-THAN  *R  16) 

(GREATER-THAN  *?  9) 

(  STRENGTH  ♦SIGHT  VS  IT.) 

( SPIRT  *SIGHT  *SPD ) 

( ♦NOT*  (GREATER-THAN  *S?D  2 ?.)  ,) 

ACTIONS 

( v*OR*  (TYPE  *X  FISHING) 

(TYPE  *X  PATROL) 

( TYPE  *X  SUB) ) ) 

CON?  .IE  PR  INFORM 

If  the  range  cf  a  weak  sighting  is  between  9  and  15,  and 
the  speed  is  less  then  22,  then  the  contact  is  possibly  e 
sub  cr  a  patrol  or  a  fishing  vessel.  (.15)  ) 

SMALL-CRAFTS 

(CONDITIONS  ((CONTACT  *WKC) 

(SIGHTING  *VHG  ♦SI) 

(NOT-iIP.ST  ♦SI) 

(SOURCE  *S1  RADAR) 

(RANGE  *S1  ♦RANGE) 

(LESS -THAN  ♦RANGE  16) 

(GREATER-THAN  *RANGE  S) 

(STRENGTH  *S 1  VE AH  ) 

(SFZED  *S1  ♦SPEED) 

(GREATER-THAN  ♦SPEED  22)) 

ACTIONS 

((♦OR*  ' TYPE  *VHC  SUB) 

(TYPE  *VHC  PATP.CL))) 

CON?  .3  PRI Nr  CRN 

"if  the  range  of  a  weak  radar  sighting  is  between  S  end  16, 
and  the  speed  is  greater  than  22,  then  the  contact  is 
possibly  a  sub  cr  a  patrol.  (.3)") 

SMALL-CRa?T4 

(CONDITIONS  ((CONTACT  ♦UKXNCVN) 

(SIGHTING  *UNKNCW !<  *SIGKTlNGi; 

( LAND-DIST  *S IGET 1 NG1  *DIST) 

(SOURCE  *S I GHTI NG 1  RADAR) 

(RANGE  *SIGHTING1  *P.ANGE ) 

(LESS-THAN  ♦RANGE  9) 

(GREATER-THAN  *RANGE  3} 
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(STRENGTH  "“S  IGHTIwG  1  W ZAK) 
i  LESS -TRAN  *DIST  52)) 

ACTION  S 

((*0?.*  (TYPE  "“UNKNOWN  SUB) 

(TYPE  "“UNKNOWN  SHORE-PATROL) 

(TYPE  "“UNKNOWN  PLEASURE) 

(TYPE  "“UNKNCW  N  COMMERCIAL) 

(TYPE  "“UNKNOWN  LANDING))) 

CONE  .1  PRINFGRN 

'if  the  range  cf  a  weak  sighting  is  between  3  and  9,  and  tae 
distance  from  land  of  the  sighting  is  less  than  52,  then  the 
vessel  may  be  a  sub,  a  patrol,  atleasure  craft,  a  landing 
craft,  or  a  commercial  craft.  (.1)'') 

SNAIL-CRAET3 

(CONDITIONS  ((CONTACT  "“UNKNOWN) 

(SIGHTING  "“UNKNOWN  "“SIGHTING) 

(LAND-DIST  "“SIGHTING  *DIST) 

(SOURCE  "“SIGHTING  RADAR) 

(RANGE  "“SIGHTING  GRANGE) 

(LESS-THAN  GRANGE  S) 

(GREATER-T5AN  GRANGE  3) 

(STRENGTH  "“SIGHTING  WEAK) 

(GREATER-THAN  *DIST  52)) 

ACTIONS 

( l TYPE  * UNKNOWN  SUB)) 

CONE  .35  PRINFCRN1 


If  the  range  cf  a  vea*  radar  sighting 
and  the  sighting  is  further. than  50  miles 
the  contact  is  a  sub.  (.35)’) 


weaA  radar  sighting  is  between  3  and  9, 
urther  than  50  miles  from  land,  then 


S^ALL-CRAF^Z 

(CONDITIONS  ((CONTACT  "“UNKNOWN) 

(SIGHTING  "“UNKNOWN  "“SIGHTING) 

(NOT-ilRST  "“SIGHTING) 

(SOURCE  "“SIGHTING  RaDAR) 

(RANGE  "“SIGHTING  "“RANGE) 

{ LESS-TEAN  "“RANGE  3) 

(STRENGTH  "“SIGHTING  WEAK) 

(SPEED  "“SIGHTING  "“SPEED) 
l  "“NOT*  (GREATER-THAN  "“SPEED  3,.; 

ACTIONS 

(("“OR*  (TYPE  "“UNKNOWN  DEBRIS) 

(TYPE  "“UNKNOWN  SUB) 

( TYFE  "“UNKNOWN  BUOY))) 

CONE  .12  PRINFORN 

’’if  the  weak  radar  sighting  is  not  known  to  be  moving  faster 
than  3  knots,  then  the  contact  is  either  a  buoy,  a  sub,  or 
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debris  .  (  .12  ;  ' ) 

SM  AL1-CRAJT1 

(CONDITIONS  ((CONTACT  ^UNKNOWN) 

(SIGHTING  *UNENCWN  *S  IGrTINC- ) 

(N'CT-i  IRST  ^SIGHTING) 

(SOURCE  ^SIGHTING  RaDAP.  } 

(RANGE  ^SIGHTING  GRANGE) 

(LaSS-TEAN  GRANGE  3) 

(STRENGTH  ^SIGHTING  WEAK) 

(SPIEL  ^SIGHTING  *SFEED } 

(GREATER-THaN  ’■■‘SPEED  3)) 

AC  TION  S 

((TYPE  ^UNKNOWN  SUB) 

(MCDZ  *UN KNOWN  PERISCOPES  CP.%  SNORKEL)) 

CCNF  .6  PRINFCRM 

"if  the  weak  radar  si^htin^  is  movies:  at  greater  than  3 
knots,  then  the  contact  is  a  su'd  in  either  periscope  or 
s ncrkel  mode .  (.£)') 

ID-LANE 

(CONDITIONS  ((SIGHTING  *SHI?  "■‘SIGHTING) 

(M2RCEAKTLANE  #LANE ) 

(PLATEGBM  *SBIP) 

(LOCATION  *IAMF  *LANZL0C  ) 

(POSITION  ^SIGHTING  *PCS  ) 

(IN-LANE  *LAML0C  *?C$  )  ) 

ACTIONS 

(  (  INS  IDE-A-MERCEANTLANE  *SIGHTING, 

(^REPORT*  *SHIP  "  was  sighted  in  the  merchant  lane  "  ^LAN 
CCNF  1.2  PRINFCRM 

"if  a  ship  is  sighted  within  some  me  rcha.o  tlane ,  then  record 
that  it  is  inside  that  lane.) 

I N  5  t  j 7 — g  t  0  R M 
(CONDITIONS  ((STORM  *STCRM) 

( PLAT FORM  *SSI?) 

( ♦UNLESS*  i I  DENT  lx  I  El  *SHIP}) 

(LOCATION  *STCRM  *STMLCC) 

(SIGHTING  *SHIP  ^SIGHTING) 

(POSITION  ^SIGHTING  *PCS  ; 

(INSIDE  *?OS  *STMLCC )  / 

ACT  I C  N  S 

('i1ype>’*seip  merchant ) 

(♦REPORT*  *SH I  P  "  was  sighted  inside  "  *STCRM)) 

CCNF  -.25  ?R INFORM 

’If  a  ship  is  sighted  inside  a  storm,  then  the  ship  ra;:  r.ot 
he  a  merchant.  (.25)’) 
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clcse-pcpu? 

(CONDITIONS  ( ( CONTACT  *SLIP) 

(FIRST-SIGHTING  *3  HI?  ^SIGHTING) 
l SANG £  * SIGHTING  ♦RANGE) 

(LESS -THAN  GRANGE  12)  ) 

ACTIONS 

((type"*sei?  kerchant); 

CON?  -.2  PRIKFGRK 

’if  the  first  sighting  cf  a  ship  is  within  12  nn,  then  it 
ray  not  be  a  merchant.  (.2)") 

DISTANT-POPUP 

i CONDITIONS  '(CONTACT  *SEIP) 

( FIRST-SIGHTING  *SEI?  ^SIGHTING) 

(RANGE  ^SIGHTING  *RANGE) 

(GREATER-TFAN  ♦RANGE  30}) 

ACTIONS 

((TYPE  *SSIP  MERCHANT)) 

CONE  -  .2  PRINFCRN’ 

"if  the  range  of  the  first  sigating  is  greater  than  SO  nm, 
then  the  ship  might  net  he  a  merchant.  (.2)’’) 

COURSE-CHANGED 

(CONDITIONS  ( (CONTACT  ♦SHIP) 

(SIGHTING  ♦SHIF  ♦SIGETINGl) 

(NOT-FIRST  ♦SIGETINGl  ) 

(NOT-LAST  ♦SIGETINGl) 

(SUCCESSOR  ♦SIGETINGl  *S IGFTING2 ; 

(COURSE  *S IGHT I NG 1  ♦C0URSE1) 

(COURSE  ♦SIGHTING2  ♦C0URSS2) 

(♦UNLESS*  (  RCUGELY-TEE-S APE- COURSE- AS  *C0URSE1 

♦CCURSE2 ) ) ) 

ACTIONS 

((TYPE  * S E I P  NERCHANT)) 

CONE  -.3  PRINECRP 

'if  the  course  has  changed,,* ignif icantly ,  then  the  sighting 
may  not  he  a  merchant.  (.3)") 

S  prr- hangfd 

CONDITIONS  ((CONTACT  *SEIP) 

(SIGHTING  *SHIP  ♦SIGHTING) 

(NOT-FIRST  *SIGHTING ) 

(NOT-LAST  *S  IGETING  ) 

(SUCCESSOR  *SIGHTING  *SIGHTING2) 

(SPEED  *S IGHTI NG  *SPEED1) 

(SPEED  *S IGETI NG2  *SPEED2) 

(♦UNLESS*  (RCUGSLY-TEE-SAPZ-SPZED-AS  *S?EED1 
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♦SPEED2 ) ) ) 

ACTIONS 

((TYPE  *seip  merchant;; 

CONE  -  .3  PR INFORM 

If  the  speed  has  changed, significantly ,  then  the  sighting 
fray  not  he  c  merchant.  (.3)”) 

JASTER-TH  A  P  -A-M  F?-CH  AN  T 
(COALITIONS  ((CONTACT  *SEIP) 

(SIGHTING  *SHIP  ^SIGHTING) 

(NCT-FIRST  ^SIGHTING) 

(SPEED  ^SIGHTING  *SPEED) 

(GREaTIR-THAN  *SPEID  2d)) 

ACTIONS 

((TYPE  *SKIF  MERCHANT)) 

CCNF  -.2d  PPINFCRM 


'If  the 
rerche  r.t 


sneed  is  greater  than  25  knots 
(  .25 )  ) 


then  it  is  net  a 


S  LC'VER-THAN-A-MIRCSANT 

.CONDITIONS  ((CONTACT  *SEIP) 

(SIGHTING  *SEIF  ^SIGHTING) 
(NOT-FIRST  ^SIGHTING) 

(SPEED  ^SIGHTING  ♦SPEED , 

(LESS -THAN  ♦SPEED  9)) 

ACTIONS 

((TYPE  ♦SHIP  MERCEANT)) 

CCNF  -.Id  PHI N FOR A 

If  the 
me  rche  nt , 

MATCH -PLAT 

CONDITIONS  ((FIRST-SIGHTING  ♦PLATA  ♦SGTl) 
(NCT-LAST  ♦SGTl) 

(SUCCESSOR  ♦SGTl  ♦SGTIS  ) 
(LAST-SIGHTING  ♦PLATE  ♦SGT2) 
(♦UNLESS*  (OWNSHIP  *?LAT1)) 
(♦UNLESS*  (OWNSHIP  *PLAT2  )  ) 
(♦NOT*  (SAME-AS  *PLAT1  *PLAT2  ) ) 
(FOSITION  *SGT1S  *PCS1S) 

(SPEED  *SGT1S  *S PD  1 ) 

(POSITION  *SGT1  *POSl  ) 
l COURSEFRCM  *POSl  *PCS1S  *CES1) 
(TOS  *SGT 1  *T1 ) 

(POSITION  *SGT2  *P0S2  ; 

(TOS  *SGT2  *T2) 

(LESS-THAN  *T2  *T1) 

( COURSEFRCM  *P0S2  *POSl  *CRS2) 


M 


(SPIIDFRC!''  *PCS2  *T2  *PCS1  *T1  *SPL2) 

(  ROUGHLY-THE-S  AN'E-CGURS  Z-A.S  *CRS1  *CRS2) 
(R0UGKLY-TH2-SAME-SPZZL-AS  *SPL1  *SFL2)) 

ACTIONS 

( (ALIAS  *?1AT2  *PLAT1)  ) 

CON?  . t  PRINrORfh 

"if  tne  course  and  speed  of  two  sightings  are  roughly  the 
sane,  and  if  one  sighting's  position  would  te  the  ether 
sighting's  extrapolated  position,  then  the  two  sightings  ere 
of  the  sane  vessel.  (.5)  ) 

CUTS IE2-ALL-LANES 

(CONDITIONS  ((SIGHTING  *SEIP  ^SIGHTING) 

(^UNLESS*  (IDENTIFIED  -SHI?)) 
iPLATiORM  *SE I F ) 

(♦UNLESS*  (MEDIUM  *SHIP  AIR)) 

(♦UNLESS*  (INSIDE-A-f'ERCHANTLANE  ^SIGHTING  ) )  ) 
ACTIONS 

( (TYPE  *SHI P  MERCHANT)  ) 

CON  F  -,?6  PRINFCRh 

"if  a  sighting  is  outside  all  merchant  lanes,  then  the 
vessel  right  not  be  a  rerchant.  (.OS)") 


STOF 


Apr  cNC  I X 


rzssno: 


This 

is  the  ressa^e 

file 

which  STANrZR.2  will 

read,  in 

sequential  order 

{vice 

tine  order),  during  the 

execution 

of  the  scenario. 

It  was 

built  by  putting  >ZS 

generated  reports  into  STA.hNEP.2  readable  ?orr. 


(BELKNAP  64.316  -28.97  2) 
(BERKELEY  RALAR  64.2  -29.286  22) 


(KNOX 

RADAR  64 

.3  -28.815  23) 

( ivSRl 

RADAR  64 

i  "*  n  '■ 

*  u  u  lu  *  w  l  y 

(MSR2 

RALAR  64 

.832  -29.157  0 ) 

( £221 

IW  270.0 

K57  24) 

(2021 

EXTERNAL 

274.2  BERKELEY 

54.3  -29.10  25) 

( E222 

EW  92. 2  i 

K3  12) 

(1222 

EXTERNAL 

90.0  BERKELEY 

64.350  -29.15  14) 

(1221 

EXTERNAL 

267.2  3ERKELEY 

64.333  -29.133  13) 

( BELKNAP  64.4  • 

-28.97  20) 

(KNOX 

HA CAR  64 

.367  -25.734  23) 

(1221 

EXTERNAL 

263.2  BERKELEY 

54.367  -29.157  22' 

(  2  2  Z  2 

EXTERNAL 

92.2  BERKELEY 

64.367  -25 .167  22  ) 

(S234 

EXTERNAL 

64  71c  -29.332 

26) 

( S  224 

EXTERNAL 

54.582  -29.425 

35) 

(U201 

EXTERNAL 

64.252  -26.232 

32  ' 

(rZLKNAr  64.45! 

;  -28 .97  42 ) 

(BERK? 

'LEY  RADAR  64.4  -29.267 

42) 

t  KNCX 

RALAR  64 

.432  -28.622  42 

/ 

(S005 

EXTERN AL 

64.422  -22.25 

42) 

( S  224 

EXTERNAL 

64.567  -29.422 

42  ) 

'  '•'202 

EVt  242.2 

LGNKA  41) 

( 

EXTERNAL 

228.0  BERKELEY 

64.433  -29.267  42 ' 

l «.  2  1 

EXTERNAL 

64.216  -28.52 

42  ) 

-  E223 

EXTERNAL 

259.2  BERKELEY 

64.467  -29.32  46 ) 

(5006 

EXTERNAL 

64.467  -30.216 

45  ) 

(3  224 

EXTERNAL 

54.532  -29.457 

45) 

(S  005 

EXTERNAL 

64.485  -29.964 

5e : 

(S034 

EXTERNAL 

64.616  -29.50 

50) 

( U  2  2 1 

EXTERNAL 

64.350  -28.646 

52  ) 

(  U  2  3 1 

EXTERNAL 

64.350  -28.646 

50  ' 

(E224 

EXTERNAL 

47.0  KNCX  64.5 

-28.552  57' 

( S  226 

EXTERNAL 

54.846  -29.333 

59  ) 

f  E004 

EXTERNAL 

47.3  KNOX  64.5 

-28.534  59) 

1  SELENA*  64. £70  -26.5?  62) 

UIBKZLZY  RADAR  64.5  -25.367  62) 

(KNOX  RADAR  64.5  -26.534  62) 

(E222  EXTERNAL  22 5.2  BERKELEY  64.524  -29.262  65 
(S225  EXTERNAL  64.52%  -25.62?  65) 

(5224  EXTERNAL  64.572  -29.6?  65) 

( S206  EXTERNAL  54.622  -29.233  65) 

(E234  EXTERNAL  46.2  XNCX  64.551  -26.465  72) 
(SZ22  EXTERNAL  227.2  BERKELEY  64.572  -29.422  75 

(U221  EXTERNAL  64.267  -26.752  75) 

(522?  EXTERNAL  64.76c  -27.556  74) 

(5205  EXTERNAL  64.572  -29.616  75) 

(5226  EXTERNAL  64.267  -29. 549  75) 

(BELKNAP  64.622  -25.57  62^ 

(BERKELEY  RADAR  64.57  -29.222  62) 

(KNOX  RADAR  64.566  -26.422  62) 


APPENDIX  X 


SCENARIO 


This  is  tae  scenario  which  will  he  ;;eneratec.  t y  the 
Warfare  Environmental  Simulator  (WES/  program,  resident  in 
tne  TEMX  system.  at  NCSC,  San  Die«;c,  California.  This  file 
sets  initial  ship  identity,  position,  course  ar.d  speed, 
and  a  basic  file  of  orders  to  be  carried  out  by  each  sice. 
The  criers,  which  represent  contingency  plans,  are 
executed  immediately  following  the  initialization  cf  VASSAL 
an  i  the  entry  cf  PLATE?,  which  is  the  user/interact  i v? 
program  for  *IS. 


AC  P.TH 
YES 
YES 
SHI? 

1.1  2ERXL  EERX 

N22221  AnAA  922 
54-1SN  29-25*  232  15 

SHIP 

1 .2  KNOX  XNCX 
1.22222  £2 B3  921 

64- lEtl  2£-52*  ZZZ  It 
SHIP 

1.3  2ELXN  EPIXA 
N23223  CCCC  922 

54- 1SN  2£-5S¥  222  15 

SHIP 

7.1  TOROS  YPES2 
Nlllll  lill  924 

55- 2LH  29-Z2H  245  1? 

SEIF 

7.2  SHORT  KASE 
N 11112  FPFF  925 

65- 12N  29 -22'*  162  22 

SHIP 
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7.3  BURL  KCTLI 

M 1 1 1 3  GGGS  926 
cc-uln  27-20*  ; 

SHIP 

7.4  VAZi.Y  EIOPI 

N 1 1 1 1 4  hi: HE  92? 
64-22N  32-23','.’  : 

SHIP 

7  •  c  S  i  C N  9  CHAR*. 
Mil  15  I  III  92c 
5-*— 22  M  27—52#  < 

SHIP 

7.6  MRl  KAZ5X 
N22221  922 

f4-lcN  20-16'*  ; 

SLIP 

?  .  7  Y ER2  KAZBi 
N  2  2  2  2  2  LLLL  921 
64-32N  29-12W  i 

SHI? 

7.6  MR2  EAZEK 
N 2 2 2 2 2  MM  £22 
€4-1 £N  30-20'#'  5 

CRISES 

PLUZ  PLAN  aIPA 


PCR 

BEIX.%  RZFCRT  ALL  ALL  TIM 

1  999 

FOR 

PERKL  REPORT  ALL  ALL  TIM 

1  =-99 

TC? 

KNOX  REPORT  aLL  ALL  TIM 

1  999 

FLAC 

*  a  i'  A  a  f.  z.  R  o  4  —  1 0  N  30  —4  €  * 

^  T  v  7  1 

a  i.  X 

939 

FLAG 

I  A  PASSER  €4-22:.  29-20', v 

T  ^  y  t  « 

dd9 

FLAC 

E  A  LARKZF  cS-lSN  2 6-0*.'# 

Mv:  1 

c  eg 

C  RANG  I  PLAN  r.LF  A 

7  OP. 

SHORT  REFCRT  ALL  SURFACE 

t:m  1 

399 

7CH 

VAZNY  COURSE  04c  TIM  4c 

JC  R 

BURL  SPIEL  20  TIM  53 

i  CR 

YCRC3  REPORT  EMM  A  IF.  TI 

►  ■  -7  T 

^  4 

-  V  A4 

HURL  REPORT  ALL  r.LL  TIM 

2c  bli 

FOR 

VAZNY  REPORT  ALL  SURFACE 

TIM  1 

999 

PC?. 

S S G N »  REFCRT  ALL  SURFACE 

TIM  1 

c  24 

~  C  f» 

MRl  RZFCRT  ALL  SURFACE  I 

:m  1 

999 

r  T  C  rf  t7rro7»'!',f  c 
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